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various  measures  correlate  with  one  another.  In  this  j 

dissertation,  I  examined:  (a)  the  true  score 
intercorrelations  between  the  various  dimensions  of  job 
performance;  and  (b)  test  whether  a  hierarchical  model 
explains  the  true  score  correlations  between  the  different 
job  performance  measures. 

Psychometric  meta-analysis  was  used  to  establish  the  true 
score  correlation  between  the  conceptually  distinct  measures 
of  job  performance.  Twenty  five  conceptually  distinct 
measures  wfere  identified  by  grouping  the  list  of  job 
performance  measures  used  in  the  literature  into  similar 
groups.  The  conceptually  distinct  measures  were  further 
grouped  into  five  groups.  An  extensive  search  of  the 
literature  was  conducted  to  locate  all  studies  that  reported 
a  correlation  between  the  job  performance  measures.  A  total 
of  2,641  correlations  based  on  283  independent  studies  were 
found. 

A  two  level  hierarchical  model  in  which  all  job 
performance  measures  were  hypothesized  to  load  on  a  general 
factor,  and  a  three  level  hierarchical  model  were  the  job  : 

P^^^^^Tnance  measures  were  hypothesized  to  load  on  one  of  the 
five  group  factors  (which  in  turn  were  hypothesized  to  load 
on  a  second  order  general  factor) ,  were  tested. 

Results  indicated  a  positive  manifold  of  true  score 
correlations  across  the  twenty  five  dimensions  of  job  ■ 
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performance.  Confirmatory  factor  analyses  indicated  the 

presence  of  a  general  factor  in  both  the  two  level  and  the 

three  level  (involving  group  factors)  hierarchical  models  of 

job  performance.  The  magnitude  of  the  general  factor  was 

larger  in  the  three  level  hierarchical  model.  Further,  the 

residuals  were  smaller  in  the  analysis  of  the  three  level 

hierarchical  model  (i.e.,  the  three  level  model  better 

explained  the  true  score  correlations  between  the  various 

dimensions  of, job  performance  than  the  two  level  model) . 

I 

Implications  of  the  findings  for:  (a)  theories  of  job 
performance,  (b)  forming  composites  of  job  performance 

measures,  |c)  differential  prediction,  and  (d)  convergent 

! 

validity  of  the  different  sources  of  performance  evaluation, 
are  discussed. 
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CHAPTER  I 

INTRODUCTION 

A  theory  is  an  explanation  of  a  phenomenon  in  terms  of 
the  processes  that  cause  it  (Hunter  &  Schmidt,  1990a,  ch. 
1).  The  development  of  a  theory  is  an  important  task  in 
all  sciences  (Campbell,  1990a;  Schmidt,  1992),  and  an 
explanation  of  a  phenomenon  in  terms  of  the  processes  that 
cause  it  involves  identifying  functional  relations  between 
constructs.  All  theories  in  science  mainly  concern 
statements  about  constructs  rather  than  specific, 
observable  variables  or  measures  (Nunnally,  1978,  p.85). 

An  important  construct  that  is  widely  used  in  many 
theories  in  industrial /organizational  psychology, 
organizational  behavior,  and  human  resources  management 
(personnel  selection,  training,  and  performance  evaluation) 
in  general,  and  personnel  selection  in  particular,  is  the 
construct  of  job  performance.  Job  performance  is  an 
important  dependent  variable  in  industrial/organizational 
psychology  (Schmidt  &  Hunter,  1992).  To  date  most 
researchers  focusing  on  the  construct  of  job  performance 
have  confined  themselves  to  particular  situations  and 
settings  with  no  attempt  to  generalize  the  findings.  Also 
there  has  been  an  emphasis  on  prediction  and  practical 
application  rather  than  explanation  and  theory  building. 
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The  consequence  of  these  two  trends  has  been  a 
proliferation  of  various  measures  of  job  performance  in  the 
literature  (Smith,  1976).  Virtually  every  measurable 
individual  differences  dimension  thought  to  be  relevant  to 
the  productivity,  efficiency,  or  profitability  of  the  unit 
or  organization  has  been  used  as  a  measure  of  job 
performance.  Absenteeism,  productivity  ratings,  violence 
on  the  job,  and  teamwork  ratings  are  some  examples  of  the 
variety  of  measures  used. 

A  question  that  remains  largely  unaddressed  is  the 
extent  to  which  the  various  measures  of  job  performance 
correlate  with  one  another.  The  continuing  exploration  of 
the  wider  implications  and  powers  of  meta-analytic  methods 
have  focused  the  attention  of  researchers  from  situational 
specificity  hypotheses  and  narrow  focus  on  prediction  of 
30b  performance  to  explaining  and  understanding  the 
psychological  processes  underlying  and  determining  job 
performance.  Meta-analytic  methods  have  made  possible  the 
estimation  of  population  correlation  matrices  between  the 
various  dimensions  of  job  performance.  But  meta-analysis 
can  go  beyond  just  quantitatively  clarifying  and 
summarizing  relationships  between  the  various  dimensions  of 
job  performance.  Along  with  the  application  of 
confirmatory  factor  analysis  and  latent  variable  analysis, 
we  can  now  test  hypotheses,  theories,  and  conceptual 
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explanations  for  the  broad  pattern  of  relationships 
established  by  meta-analysis  across  the  various  dimensions 
of  job  performance. 

In  this  dissertation,  I  examine:  (a)  the  true  score 
intercorrelations  between  the  various  dimensions  of  job 
performance;  and  (b)  whether  a  hierarchical  model  (to  be 
elaborated  later)  explains  the  true  score  correlations 
between  the  different  job  performance  dimensions. 

Importance  of  Establishing  the  Intercorrelations 

Why  is  it  important  to  estimate  the  true  score 
correlations  between  the  various  job  performance 
dimensions?  Establishing  the  true  score  correlations 
between  the  various  dimensions  of  job  performance  is 
important  because  of  the  implications  for  forming 
composites  as  well  as  for  the  search  for  common  predictors 
of  different  dimensions  of  performance.  High  correlations 
imply  that  the  different  dimensions  of  job  performance  have 
convergent  validity  and  combining  them  as  one  composite 
measure  will  be  theoretically  meaningful  (Schmidt  &  Kaplan, 
1971)  .  Further,  if  the  dimensions  are  highly  correlated, 
then  regardless  of  the  particular  dimension  used  as 
criterion,  the  same  test  battery  (or  set  of  predictors) 
will  be  chosen.  That  is,  differential  prediction  will  be 
unlikely,  thus  encouraging  the  search  for  common  predictors 
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that  can  be  used  as  the  basis  for  developing  theories  of 
work  behavior  and  human  motivation. 

Moreover,  examining  the  true  score  correlations  between 
the  various  dimensions  of  job  performance  is  an  essential 
first  step  in  testing  and  understanding  the  latent 
structure  of  job  performance.  High  intercorrelations 
between  the  various  dimensions  generally  support  the 
conclusion  that  the  dimensions  have  convergent  validity 
(i.e.,  they  measure  the  same  construct)  and  that  the 
different  dimensions  will  have  high  factor  loadings  on  one 
common  factor.  Examining  the  true  score  intercorrelations 
IS  an  essential  first  step  in  testing  the  latent  structure 
of  job  performance  because  mathematically  it  is  possible 
for  two  measures  to  have  very  low  correlation,  yet  have 
high  factor  loadings  on  a  common  underlying  construct. 
Nunnally  (1978,  p.  368)  gives  an  example  of  two  variables 
each  with  factor  loadings  of  .50  on  the  first  factor,  but 
with  +.50  and  -.50,  respectively  on  the  second  factor;  a 
pattern  of  loadings  that  stem  from  a  correlation  of  zero 
between  the  two  variables.  This  underscores  the  importance 
of  first  examining  the  intercorrelations,  before  testing 
for  the  latent  structure  of  job  performance  using 
confirmatory  factor  analysis.  That  is,  the  first  and  most 
important  step  in  modeling  job  performance  is  to  focus  on 
the  true  score  intercorrelations  between  the  various 


dimensions  of  job  performance.  Using  these  true  score 
intercorrelations,  we  can  then  test  competing  models  of  job 
performance  hypothesized  to  explain  the  latent  structure  of 
job  performance. 

However,  high  true  score  intercorrelations  indicating 
that  two  variables  measure  the  same  construct  does  not 
imply  that  the  two  variables  are  conceptually  similar.  Two 
variables  can  correlate  highly,  measure  the  same  underlying 
construct,  and  yet  be  conceptually  distinct.  For  example, 
both  verbal  and  quantitative  abilities  measure  General 
Mental  Ability  and  correlate  highly,  but  are  conceptually 
distinct.  Another  example  is  the  Raven  Progressive 
Matrices  test.  The  internal  consistency  measures  are  in 
the  .90s  and  factor  loadings  indicate  a  highly  homcreneous 
general  factor  (the  factor  loadings  on  perceptual, 
numerical  and  spatial  factors  are  negligible)  but  the  items 
are  conceptually  distinct.  Thus,  though  high 
intercorrelations  between  the  measures  do  not  imply  that 
the  measures  are  conceptually  similar,  researchers  who 
advocate  that  two  highly  correlated  measures  need  to  be 
considered  separately  should  provide  the  justification  and 
rationale  for  keeping  them  separate.  The  key  point  to  note 
is  that  there  are  different  levels  of  analysis.  Two 
measures  can  be  different  on  one  level  but  interchangeable 
on  another  (Hunter  &  Schmidt,  1990a,  p.  481  &  pp.  516-524) . 
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The  Latent  Structure  of  Job  Performance 

Why  is  it  important  to  investigate  the  latent  structure 
of  job  performance?  The  maturation  of  a  scientific  field 
is  reflected  in  the  relative  effort  expended  on  theoretical 
analysis  and  attempts  to  explain  and  understand  the 
underlying  processes  and  mechanisms  as  compared  to  the 
effort  expended  on  developing  practical  applications.  The 
insecurity  of  a  nascent  field  in  the  community  of  sciences 
prompts  an  emphasis  on  developing  practical  applications 
(Schmidt  Sc  Kaplan,  1971)  .  Once  the  usefulness  of  a 
profession  is  accepted  by  the  society,  researchers  indulge 
in  the  luxury  of  theory  building.  With  increasing  emphasis 
in  I/O  psychology  on  theory  building  (Campbell,  1990a; 
Schmidt,  1992),  there  is  an  urgent  need  to  establish  the 
nomological  network  of  constructs  that  need  to  be  included 
in  theories  of  work  behavior  and  human  motivation.  Job 
performance  is  perhaps  the  most  important  construct  and 
there  is  a  need  to  examine  the  latent  structure  of  job 
performance  to  explicate  that  construct. 

To  answer  the  question  whether  the  different  measures 
of  job  performance  are  manifestations  of  the  same 
underlying  construct,  two  lines  of  evidence  are  relevant. 
First,  a  positive  manifold  (a  predominance  of  positive 
values  in  the  intercorrelation  matrix)  of  true  score 


correlations  among  the  different  measures  would  indicate 
the  presence  of  a  common  underlying  construct.  A  positive 
manifold  of  true  score  correlations  between  the  various 
measures  indicates  the  presence  of  a  general  factor,  the 
magnitude  of  which  indicates  how  well  the  various  measures 
"go  together"  in  empirical  investigations.  Second, 
evidence  that  the  various  measures  have  similar 
relationships  with  other  well  defined  constructs  would 
strengthen  the  inference  that  the  various  measures  are 
manifestations  of  a  common  construct.  More  specifically, 
if  other  constructs  (e.g..  General  Mental  Ability  and 
Conscientiousness)  can  be  shown  to  be  the  common  causal 
agent  of  the  many  different  measures  of  job  performance,  we 
can  infer  that  there  is  a  common  underlying  construct 
across  the  different  measures  of  job  performance  (i.e.,  the 
construct  of  job  performance  is  explicated  as  referring  to 
the  set  of  different  measures  of  job  performance  included 
in  the  analyses) .  The  first  line  of  evidence  focuses  on 
the  internal  structure  of  the  construct  of  job  performance 
whereas  the  second  line  of  evidence  explores  the  cross¬ 
structure  of  measures  of  job  performance  with  measures  of 
other  constructs.  In  this  dissertation,  I  establish  the 
true  score  intercorrelations  between  the  various  measures 
of  job  performance,  and  test  whether  a  hierarchical  model 
of  job  performance  can  explain  the  intercorrelations. 
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A  distinction  is  made  in  this  dissertation  between  an 
economic  construct  and  a  psychological  construct.  An 
economic  construct  of  job  performance  focuses  on  the 
economic  end  products  of  job  behaviors  and  not  on  the 
behaviors  themselves.  All  performance  measures  can  be 
expressed  in  economic  units  (i.e.,  dollars  and  cents)  and 
much  of  recent  research  in  utility  analysis  (e.g., 

Judiesch,  Schmidt,  &  Mount,  1992)  has  focused  on 
identifying  the  cognitive  processes  underlying  the  scaling 
of  performance  in  different  metrics  to  a  common  dollar 
metric . 

The  ubiquitous,  general  economic  factor  in  the  various 
measures  of  job  perfor.mance  is  the  basis  for  combining  the 
measures  into  an  _  .-erall  composite  measure  regardless  of 
their  intercorrelations.  In  fact,  Schmidt  and  Kaplan 
-971)  had  argued  that  even  when  multiple  independent 
measures  have  been  used,  for  purposes  of  practical  decision 
making,  the  assessments  of  individuals  on  various 
dimensions  must  be  combined  into  an  overall  composite 
measure.  Failure  to  develop  explicit  methods  of 
combination  merely  relegates  the  combination  of  measures  to 
one  of  ad  hoc  subjectivity  on  the  part  of  the  decision 
maker.  All  decisions  in  the  real  world,  such  as  selection, 
are  based  on  an  overall  composite  measure.  A  similar  idea 
underlies  the  arguments  of  writers  (Krug,  1961;  Stone  & 


Kendall,  1955;  Tiffin  &  McCormick,  1958)  favoring  the 
composite  measure.  Terms  such  as  ultimate  criterion,  job 
success,  standard  of  reference  or  yardstick  to  evaluate 
overall  performance  across  all  measures  of  job  performance, 
reflect  the  theme  of  an  underlying  economic  construct 
across  the  different  measures. 

A  psychological  or  behavioral  construct,  on  the  other 
hand,  examines  the  homogeneity  of  the  various  measures  of 
job  performance.  A  psychological  construct  addresses  the 
question  of  the  extent  to  which  the  various  measures  “go 
together"  or  cluster  together.  Basically,  a  psychological 
construct  focuses  on  the  common  variance  represented  by  the 
true  score  intercorrelations  between  the  different  measures 
of  the  construct  under  investigation,  and  examines  how  this 
common  variance  can  be  defined  so  as  not  to  cause  confusion 
with  the  definition  of  the  common  variance  of  a  different 
set  of  behavioral  measures  (representing  another 
psychological  construct) .  The  focus  in  this  dissertation 
is  not  on  the  common  underlying  economic  construct,  but  on 
the  question  of  whether  there  is  a  common  underlying 
nsvcholoaical  construct  across  all  measures  of  job 
performance.  That  is,  the  focus  in  this  dissertation  is  on 
examining  the  psychological  construct  of  job  performance. 

In  investigating  whether  there  is  a  common  underlying 
psychological  construct  across  the  various  measures  of  job 


performance  a  parallel  can  be  drawn  to  the  situation  in  the 
domain  of  abilities,  where  prior  to  the  work  of  Spearman, 
the  popular  anarchic  view  postulated  that  performance  on 
any  task  is  dependent  on  its  own  specific  ability.  This 
democratization  of  abilities  (Gottf redson,  1988)  satisfied 
many  and  was  expedient  to  advocate.  Individuals  scoring 
low  on  one  ability  could  always  console  themselves  as  being 
high  on  other  abilities.  But  based  on  empirical  data 
indicating  the  positive  manifold  of  correlations  among  the 
abilities,  Spearman  (1904)  established  the  universal  unity 
of  the  Intellectual  function,  showing  that  the  various 
forms  of  mental  activity  constitute  a  stable  interconnected 
hierarchy  according  to  their  different  degrees  of 
intellective  saturation.  Spearman  tested  a  two  level 
hierarchical  model  involving  a  general  factor  (common  to 
all  forms  of  mental  activities)  at  the  first  or  highest 
level,  and  specific  factors  (specific  to  each  form  of 
mental  activity)  at  the  second  or  lower  level.  The  two 
level  theory  of  Spearman  was  found  to  be  too  simple  and 
subsequent  modifications  and  refinements  have  converged  on 
a  three  level  hierarchical  model  involving  a  general 
factor,  group  factors,  specific  factors,  and  random 
measurement  error.  In  a  three  level  hierarchical  model, 
the  general  factor  is  at  the  first  or  highest  level,  group 
factors  are  at  the  second  level,  and  specific  factors  are 
at  the  third  or  lowest  level  of  the  hierarchy. 


The  literature  in  job  performance  is  fragmented  and 
mirrors  the  state  of  the  literature  in  the  predictor  domain 
that  existed  before  Spearman  (1904) .  One  aim  in  this 
dissertation  is  to  examine  whether  the  true  score 
correlation  between  the  various  measures  of  job  performance 
can  be  explained  by  a  two  level  hierarchical  model 
involving  a  general  factor  common  to  all  measures  along 
with  specific  factors  for  each  measure.  That  is,  the 
confirmatory  model  tested  in  this  dissertation  views  the 
various  measures  of  job  performance  such  as  quality  and 
quantity  of  work  performance,  absenteeism,  turnover, 
violence  on  the  job,  and  teamwork  as  the  manifestations  of 
a  general  construct  of  job  performance.  This  hypothesis 
can  be  stated  in  factor  analytic  terms  as  follows.  There 
is  a  general  factor  common  to  all  the  measures  of  job 
performance  and  an  individual's  standing  on  a  particular 
measure  of  job  performance  is  the  sum  of  the  individual's 
standing  on  this  general  factor  plus  that  individual's 
standing  on  the  specific  factor  (specific  to  that  measure 
of  job  performance) . 

In  addition  to  testing  a  two  level  hierarchical  model, 

I  also  test  a  three  level  model.  In  a  three  level  model-- 
just  as  the  response  of  an  individual  to  a  questionnaire  or 
test  item  is  dependent  on  a  general  factor,  group  factor, 
specific  factor,  and  a  random  error  component--  the 
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standing  of  an  individual  on  any  specific  performance 
measure  (e.g.,  absenteeism)  is  hypothesized  to  depend  on 
the  general  factor  (i.e.,  overall  job  performance),  the 
group  factor  (e.g.,  the  withdrawal  behavior  of  the 
employee;  absenteeism,  tardiness,  time  theft,  turnover  all 
may  belong  to  this  group),  the  specific  factor  (i.e., 
absenteeism),  and  a  random  error  component.  That  is,  the 
hypothesized  latent  structure  postulates  a  general  factor 
at  the  highest  or  first  level,  which  causes  the  performance 
on  the  group  factors  at  the  second  level.  The  group 
factors,  in  turn,  act  as  causal  antecedents  to  the  various 
job  performance  measures.  The  general  factor  and  the  group 
factors  are  latent  variables.  At  the  third  or  the  lowest 
level  are  the  various  job  performance  dimensions,  which  are 
the  observed  or  manifest  variables. 

The  group  factors  in  the  three  level  hierarchical  model 
were  formed  based  on  a  review  of  the  job  performance 
literature.  However,  it  should  be  noted  that  the  group 
factors  identified  and  included  in  the  three  level  model 
tested  here  reflect  my  interpretation  of  the  literature. 
Alternate  three  level  models  with  a  different  set  of  group 
factors  can  be  hypothesized.  Using  the  true  score 
correlations  reported  in  this  dissertation  the  alternate 
models  can  be  empirically  tested.  That  is,  in  this 
dissertation,  I  subject  the  matrix  of  true  score 


intercorrelations  between  the  various  dimensions  of  job 
performance  to  confirmatory  factor  analysis  to  test  a  two 
level  hierarchical  model  of  job  performance;  in  addition, 
based  on  my  review  of  the  literature  I  present  and  test  one 
possible  three  level  hierarchical  model. 

An  issue  to  consider  in  testing  the  hypothesized 
hierarchical  latent  structure  of  job  performance  is  one 
that  is  predominantly  discussed  in  performance  evaluation: 
halo.  The  various  measures  of  job  performance  can  be 
broadly  grouped  as  those  involving  subjective  judgments 
(ratings  or  rankings)  or  objective  records.  The  study  of 
subjective  judgments  (either  rankings  or  ratings)  of 
individuals  on  various  measures  has  lead  researchers  to  the 
issue  of  halo  (King,  Schmidt,  &  Hunter,  1980).  Halo  has 
been  conceptualized  in  the  literature  as  either:  (a)  a 
general  impression  that  produces  excessively  high 
correlations  between  rated  dimensions  (e.g.,  a  correlation 
of  .90  between  personal  appearance  and  performance)  when 
both  the  ratings  are  provided  by  the  same  individual;  or 
(bi  the  1 diosvncratic  overall  evaluation  of  a  rater  (and 
this  idiosyncrasy  is  what  makes  interrater  agreement  lower 
than  intrarater  agreement) .  To  unconfound  the  effects  of 
halo  from  a  potential  general  factor  across  the  ratings  of 
various  dimensions  (when  testing  for  the  hypothesized 
latent  structure  of  job  performance) ,  I  used  the 
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correlations  between  (a)  job  performance  measures  based  on 
organizational  records,  and  (b)  between  ratings  of  various 
job  perform^ance  dimensions  when  the  two  ratings  being 
correlated  are  provided  by  different  raters.  The 
appropriate  reliability  distribution  to  be  used  in 
correcting  each  type  of  correlation,  along  with  an 
elaboration  of  the  issue  of  halo,  is  provided  in  the 
methods  section. 

The  outline  of  the  dissertation  is  as  follows. 

Following  this  introduction,  in  the  second  chapter,  I 
review  the  empirical  studies  examining  the 
intercorrelations  between  the  various  dimensions  of  job 
performance.  In  the  third  chapter,  the  methods  used  in 
this  dissertation  are  explained.  Information  regarding  the 
data  sources  used  as  well  as  information  on  the  artifact 
distributions  used  are  provided.  The  results  of  the  meta¬ 
analyses  are  presented  in  the  fourth  chapter  whereas  the 
fifth  chapter  discusses  the  implications  of  the  findings 
for  theory  and  practice. 
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CHAPTER  II 

LITERATURE  REVIEW 

In  this  chapter,  I  organize  a  review  of  the  job 
performance  literature  in  three  sections.  In  the  first 
section,  I  summarize  the  results  of  studies  that  examined 
the  correlations  between  the  various  dimensions  of  job 
performance.  In  the  second  section,  I  summarize  the 
results  of  previous  attempts  to  explicate  the  construct  of 
job  performance.  Finally,  in  the  third  section,  I  outline 
the  limitations  of  each  approach  for  investigating  and 
drawing  inferences  about  the  relationships  between  the 
various  dimensions  of  job  performance.  I  conclude  this 
chapter  with  a  discussion  of  how  I  addressed  these 
limitations  in  this  dissertation. 

At  the  outset,  based  on  the  specification  of  the  domain 
of  interest,  it  should  be  obvious  that  the  number  of 
studies  that  has  to  be  included  in  this  review  is  numerous. 
As  such  this  review  is  necessarily  selective  and  aims  to 
sketch  what  in  my  view  reflects  the  important  developments 
in  the  literature  on  job  performance.  In  fact,  all 
qualitative  reviews  are  highly  idiosyncratic  (except  maybe 
when  the  review  focuses  on  a  very  narrow  and  clearly 
specified  domain  where  it  is  possible  to  review  all 
studies) .  Specifically,  in  examining  the  studies  that 


focused  on  intercorrelations  between  dimensions  of  job 
performance,  no  attempt  is  made  here  to  summarize  all  the 
individual  studies  that  report  intercorrelations  between 
dimensions  of  job  performance.  (To  do  so  v/ould  be  to 
conduct  a  traditional,  qualitative,  narrative  review  before 
conducting  a  quantitative  meta-analytic  review  of  the  same 
database.)  Finally,  Austin  and  Villanova  (1992)  provide  a 
qualitative  review  of  the  measures  used  as  criteria  in 
validation  studies  (including  studies  conducted  in 
vocational  and  educational  settings  in  addition  to  studies 
conducted  in  the  workplace) ,  whereas  this  review  is  limited 
to  studies  conducted  in  workplace  settings. 

Ihi;grrel^t;iQns  .gebwg^n  .Job  Piinghsions 

Many  researchers  have  examined  the  interrelations 
between  the  various  dimensions  of  job  performance.  In  this 
section,  I  review  the  studies  that  either:  (a)  obtained 
measures  of  individuals  on  more  than  one  dimension  of  job 
performance  and  estimated  the  correlation  between  the 
dimensions;  or  (b)  analyzed  the  jobs  done  (by  critical 
incident  techniques,  task  surveys  of  incumbents,  or 
supervisors,  or  subordinates,  direct  observation  of 
incumbents  performing  the  tasks,  etc.)  and  identified  the 
different  dimensions  of  job  performance;  or  (c)  used  meta 
analytic  techniques  to  cumulate  the  results  across  studies 


reporting  correlations  between  a  particular  pair  of  job 
performance  dimensions. 

Factor  Analysis  of  Correlations 

In  a  typical  factor  analytic  study,  individuals  are 
assessed  on  multiple  measures  of  job  performance. 
Correlations  are  obtained  between  the  measures  of  job 
performance  and  factor  analysis  is  used  to  identify  the 
measures  that  cluster  together.  Reviewed  below  are  a 
sample  of  studies  utilizing  the  framework  of  factor 
analysis. 

Rush  (1953)  analyzed  9  rating  measures  and  3 
organizational  record  based  measures  of  job  performance  for 
100  salespersons  and  identified  four  facte- s  of  job 
performance.  He  named  the  four  factors  as:  objective 
achievement,  learning  aptitude,  general  reputation,  and 
proficiency  of  sales  techniques.  Baier  and  Dugan  (1957) 
obtained  measures  for  346  sales  agents  on  17  variables. 
Analysis  of  the  17  x  17  intercorrelation  matrix  provided 
support  for  a  general  factor  of  job  performance.  Various 
measures  such  as  percentage  sales,  units  sold,  tenure, 
knowledge  of  products,  loaded  on  this  general  factor. 

Ronan  (1963)  conducted  a  factor  analysis  of  11  job 
performance  measures  including  accidents  and  disciplinary 
actions  and  found  four  distinct  factors  of  job  performance 
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using  Thurstons' s  centroid  method.  One  of  the  four  factors 
reflected  the  "safe"  work  habits  of  the  individual  (index 
of  injuries,  time  lost  due  to  accidents  are  examples  of 
some  measures  that  loaded  on  this  factor) .  Acceptance  of 
authority  and  adjustment  were  two  other  factors  that  were 
identified.  Finally,  a  difficult  to  interpret  fourth 
factor  mirroring  overall  evaluation  also  emerged.  Prien 
and  Kult'(1968)  factor  analyzed  23  measures  of  job 
performance  including  errors,  productivity,  knowledge  and 
skills  ratings,  and  found  evidence  for  7  factors  of  job 
performance . 

Gunderson  and  Ryman  (1971)  examined  the  job  performance 
in  extremely  isolated  groups  (scientists  spending  the 
winter  in  Antarctica)  and  found  three  factors  of  job 
performance.  The  three  factors  they  interpreted  were  task 
efficiency,  emotional  stability,  and  interpersonal 
relations.  An  analysis  of  the  measures  they  used  brings 
home  the  point  that  anv  factor  can  be  identified  and 
interpreted  as  long  as  an  appropriate  set  of  observable 
measures  are  included  in  the  factor  analysis.  What  is 
required  is  an  a  priori  (comprehensive)  specification  of 
the  construct  domain,  estimation  of  the  true  score 
correlations  between  conceptually  distinct  measures,  and  an 
examination  (using  confirmatory  factor  analysis)  of  whether 


the  hypothesized  latent  structure  explains  the  true  score 
correlations . 

Identifying  Job  Dimensions  Based  on  Inventory  of  Tasks 

This  section  focuses  on  studies  that  examined  the 
relationships  between  the  various  dimensions  of  job 
performance  by  describing  the  domains  of  organizational 
behavior  thought  to  be  important  for  organizational 
effectiveness.  Job  analysis,  critical  incident  techniques, 
incumbent  interviews,  content  analysis  of  performance 
feedback  episodes,  observation  of  incumbents  on  the  jobs, 
and  task  surveys  are  some  techniques  employed  in  these 
studies.  Of  these,  task  surveys,  critical  incident 
techniques,  and  observation  of  incumbents  are  the  widely 
used  techniques.  The  idea  is  to  generate  a  list  of  tasks 
performed  by  the  incumbents  and  then  attempt  to  identify 
msing  factor  analysis)  from  these  task  statements  the 
different  dimensions  of  job  performance. 

Task  SyrA:gY.£ 

Task  surveys  (of  incumbents,  supervisors,  or 
subordinates)  represent  a  common  technique  used  to  identify 
the  various  dimensions  of  job  performance.  In  a  typical 
study,  task  statements  are  presented  to  the  sample  of 
respondents  (incumbents,  or  supervisors,  or  subordinates) 
who  rate  the  importance  and  frequency  of  the  tasks.  For 
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example,  Brumback  and  Vincent  (1970),  based  on  4,000  brief, 
narrative  job  descriptions  extracted  8,000  task  statements. 
A  sample  of  100  officers  from  United  States  Public  Health 
Service  (USPHS)  representing  various  positions  reviewed  the 
8,000  task  statements  and  selected '196  duty ,''v;ork,/ task 
statements  as  descriptive  of  the  various  tasks  performed  by 
incumbents.  Another  sample  of  5,000  officers  (of  which 
3,719  responded)  were  then  asked  in  a  survey  to  rate,  using 
a  checklist  format,  the  importance,  frequency  of 
occurrence,  and  amount  of  time  spent  on  each  of  the  196 
duties.  A  principal  components  analysis,  including  a 
normalized  varimax  rotation  of  the  product  moment 
correlation  matrix  of  the  196  items,  resulted  in  26  factors 
(factor  extraction  was  continued  until  the  latent  root  of 
the  last  extracted  factor  fell  below  unity  or  until  the 
latent  root  of  the  last  extracted  factor  accounted  for  less 
than  one  percent  of  the  sum  of  variance  up  to  that  point) . 

Allan  (1981)  obtained  responses  from  1,476  managers 
(1,550  were  surveyed)  as  to  the  importance  of  and  time 
spent  on  146  task  statements.  The  146  task  statements  were 
edited  by  a  panel  of  52  managers  (26  males)  and  was 
pretested  on  .40  managers  representing  both  sexes,  various 
ethnic  groups,  hierarchical  levels  and  functions.  The 
factor  analysis  identified  6  factors  that  were  sufficient 
to  describe  the  construct  of  job  performance. 
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Observation  of  Incumbents 

Komaki,  Zlotnick,  and  Jensen  (1986)  used  observational 
techniques  to  identify  the  dimensions  of  job  performance. 
Trained  observers  were  used  to  obtain  information  on  the 
time  spent  by  incumbents  on  various  tasks.  Questionnaire 
methods  used  to  identify  the  dimensions  of  job  performance 
have  employed  as  respondents  either  supervisors  or 
incumbents  or  subordinates.  Though  each  source 
(supervisors,  incumbents,  and  subordinates)  provides  a 
unique  perspective,  most  studies  have  employed  incumbents. 
Direct  observation  not  only  captures  what  incumbents  say 
and  do,  but  it  also  avoids  self-report  biases  inherent  when 
the  questionnaire  method  is  used  with  incumbents  (cf.  Rush, 
Thomas,  &  Lord,  1977).  Further  instead  of  developing  a 
taxonomy  of  tasks  post-hoc  based  on  raw  data  using  factor 
analytic  techniques  or  delphi  processes,  Komaki  et  al . 

(1986)  developed  their  factors  based  on  operant 
conditioning  theory.  Two  field  tests  using  samples  of  bank 
managers  and  theater  supervisors  identified  7  factors  of 
job  performance.  One  problem  with  this  approach  is  that 
not  all  behaviors  of  importance  to  effective  job 
performance  are  visually  observable. 
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Critical  Incident  Techniques 

Hedberg  (1989)  content  analyzed  performance  feedback 
episodes  obtained  (by  employing  interview  and  critical 
incident  methodology)  from  a  sample  of  financial  analysts 
and  project  managers.  The  results  paralleled  the  eight 
factor  latent  structure  of  job  performance  as  hypothesized 
by  Campbell  (1990b)  and  Campbell,  McCloy,  Oppler,  and  Sager 
(1992)  .  Hough  (1984)  used  the  method  of  critical  incidents 
to  develop  a  list  of  569  examples  of  effective  and 
ineffective  job  behavior  for  attorneys.  When  3 
psychologists  were  asked  to  sort  these  569  critical 
incidents  into  tentative  categories  and  describe  each 
category,  a  taxonomic  structure  based  on  11  dimensions  of 
joh  performance  resulted.  Further,  in  the  job  analysis 
phase  of  the  study  (Hough,  1984),  326  attorneys  rated  the 
time  spent  and  importance  of  various  task  activities, 
factor  analysis  of  which  yielded  six  dimensions  of  job 
performance . 

Borman  and  Brush  (1993)  collected  26  sets  of  managerial 
performance  dimensions.  Several  management  levels  as  well 
as  different  organizations  were  included.  The  26  sets  of 
managerial  performance  dimensions  resulted  in  187  non- 
redundant  task  statements  which  were  administered  to  25 
industrial-organizational  psychologists.  The  I/O 
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psychologists  were  required  independently  to  sort  the  187 
statements  into  groups  based  on  similarities  among  the 
statements.  A  187  x  187  similarity  matrix  was  constructed 
(Rosenberg  &  Sedlak,  1972)  ba..  :d  on  the  proportion  of 
judges  sorting  each  pair  into  the  same  group.  Factor 
analysis  of  this  similarity  matrix  identified  18  orthogonal 
(and  interpretable)  dimensions  of  job  performance. 

Meta-Analytic  Cumulation  Over  Pairs  of  Job  Performance 

Measures 

In  this  section,  I  review  the  studies  that  employed 
meta-analytic  techniques  to  investigate  the  convergent 
validity  between  pairs  of  job  performance  dimensions.  In  a 
typical  study,  correlations  reported  in  the  literature 
between  a  particular  pair  of  dimensions  are  obtained  and 
mulated  across  samples.  The  true  score  correlation 
computed  after  correcting  for  the  various  statistical 
artifacts  is  taken  as  an  index  of  the  convergent  validity 
between  the  two  dimensions.  The  literature  search 
identified  five  studies  that  used  this  approach. 

Heneman  (1986)  cumulated  results  across  23  studies  to 
assess  the  relationship  between  supervisory  ratings  and 
results-oriented  measures  of  job  performance.  Based  on  a 
total  sample  of  3,178,  he  found  the  corrected  mean 
correlation  to  be  .27.  Corrections  were  made  for 
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unreliability  in  the  supervisory  ratings  (used  the  value  of 
.60  provided  by  King  et  al.  [1280])  and  for  unreliability 
in  the  output  measures  (used  a  mean--across  eight  values-- 
test-retest  value  of  .63). 

Harris  and  Schaubroeck  (1988)  based  on  a  total  sample 
size  of  2,643  (across  23  correlations  or  samples)  found  the 
true  score  correlation  between  peer  and  supervisory  ratings 
of  overall  job  performance  to  be  .62.  Corrections  were 
made  for  measurement  error  in  both  peer  and  supervisory 
ratings  using  internal  consistency  measures  that  were 
substantially  higher  than  the  value  of  .60  (the  interrater 
reliability)  that  is  recommended  by  Schmidt  and  his 
colleagues.  (Using  the  value  of  .60  for  reliability, 
results  in  a  true  score  correlation  of  .71.)  Harris  and 
Schaubroeck  (1988)  also  found  no  evidence  for  the 
moderating  influence  of  rating  method.  The  true  score 
correlation  across  14  samples  involving  1,331  data  points 
using  the  dimensional  method  of  rating  was  .57.  The 
corresponding  correlation  across  15  samples  involving  1,914 
data  points  using  the  global  method  of  rating  was  . 65 . 

McEvoy  and  Cascio  (1987)  conducted  a  meta-analysis 
across  24  studies  involving  7,717  data  points  and  found  the 
true  score  correlation  between  supervisory  ratings  of  job 
performance  and  voluntary  turnover  to  be  -.28.  Reliability 
corrections  were  applied  only  to  the  performance  measures. 
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An  upper  bound  value  of  .60  for  the  interrater  reliability 
as  reported  by  King  et  al.  (1980)  was  used  to  correct  the 
performance  measures  (Glenn  &  McEvoy,  1987,  p.749).  McEvoy 
and  Cascio’s  (1987)  results  indicate  a  substantial  negative 
relationship  (poor  performers  are  more  likely  to  have 
higher  turnover  rates)  between  voluntary  turnover  and  job 
performance. 

Bycio  (1992)  summarized  the  relationship  between 
absenteeism  and  job  performance.  Across  49  samples 
involving  15,764  data  points  he  found  the  true  score 
correlation  between  supervisory  ratings  (or  rankings)  of 
job  performance  and  organizational  records  of  absenteeism 
(both  time  lost  and  frequency  measures)  was  -.29.  Based  on 
28  samples  (N  =  7,704)  that  had  used  the  time  lost  measures 
of  absenteeism  the  correlation  was  -.26  with  supervisory 
ratings  (or  rankings)  of  job  performance,  whereas  across  21 
samples  (N  =  8,060)  using  frequency  measures  of  absenteeism 
the  correlation  with  job  performance  was  -.32. 

Similarly,  the  true  score  correlation  between  non¬ 
rating  quality  of  performance  indices  and  absenteeism 
measures  (both  frequency  and  time  lost)  was  -.24  (based  on 
23  samples  and  a  total  sample  size  of  5,204).  When  the  23 
samples  were  subgrouped  into  those  using  time  lost  indices 
of  absenteeism  (11  samples  with  a  total  sample  size  of 
1,649)  and  those  using  frequency  based  measures  of 


absenteeism  (12  samples  with  a  total  sample  size  of  3,555), 
the  correlation  with  non-rating  quality  of  performance 
indices  became  -.28  and  -.22,  respectively. 

The  artifact  distribution  used  by  Bycio  (1992)  to 
correct  the  supervisory  ratings  involved  ten  interrater 
reliability  values  (the  mean  of  the  square  root  of  the  ten 
values  was  .82;  comparable  to  the  value  of  .60  provided  by 
King  et  al .  [1980])  .  For  non-rating  quality  indices,  an 

artifact  distribution  involving  4  values  were  used;  the 
mean  of  the  square  roots  of  the  four  values  was  .866.  The 
artifact  distribution  used  to  correct  time  lost  measures 
included  30  values;  the  mean  of  the  square  roots  of  the  30 
estimates  was  .733.  The  corresponding  value  for  the 
frequency  measures  (36  estimates)  was  .686. 

The  results  reported  by  Bycio  (1992)  indicate  that  both 
time  lost  and  frequency  based  measures  of  absenteeism  are 
substantially  and  negatively  related  to  both  supervisory 
ratings  (or  rankings)  of  performance  as  well  as  non-rating 
quality  of  performance  indices,  thus  raising  the 
possibility  of  a  general  factor  common  across  these  four 
measures  of  job  performance.  In  addition  to  summarizing 
the  relationships  between  the  four  measures  of  job 
performance  which  seems  to  indicate  the  presence  of  a 
general  factor,  Bycio  (1992)  also  presents  some  causal 
mechanisms  to  explain  why  a  general  factor  should  underlie 


these  measures.  Supervisory  annoyance  with  absentees  or 
disruptions  caused  to  work  scheduling  could  explain  the 
negative  relationship  between  absenteeism  and  supervisory 
ratings  of  job  performance.  The  negative  relationship 
between  absenteeism  and  non-rating  quality  of  performance 
indices  can  be  explained  either  in  terms  of  employee 
dispositions  or  by  invoking  the  notion  of  progressive 
employee  withdrawal  (Sheridan,  1985).  Hogan  and  Hogan 
(1989)  have  argued  that  poor  performance  (as  measured  by 
overall  supervisory  ratings  and  non-rating  quality  of 
performance  measures)  and  absenteeism  are  only  reflections 
of  the  delinquency  construct.  Ones,  Viswesvaran,  and 
Schmidt  (in  press)  found  evidence  that  integrity  tests  can 
predict  absenteeism,  supervisory  ratings  of  job  performance 
and  non-rating  measures  of  job  performance  (i.e.,  these 
different  dimensions  of  job  performance  have  a  common 
predictor  or  causal  antecedent) , 

Mitra,  Jenkins,  and  Gupta  (1992)  investigated  the 
relationship  between  absenteeism  and  turnover.  Based  on  33 
correlations  from  17  studies  involving  5,316  data  points 
Mitra  et  al.  (1992)  estimated  the  true  score  correlation 
between  absenteeism  and  turnover  to  be  .33.  Corrections 
were  made  for  unreliability  in  absence  measures  and  for 
dichotomization  of  absence  measures.  Further,  turnover 
measures  were  corrected  for  unequal  N‘s  (between  stayers 
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and  leavers  across  organizations) ,  but  not  for 
dichotomization  in  the  turnover  variable  (that  is,  turnover 
was  conceptualized  as  a  true  dichotomy) .  The  reliability 
distributions  for  absenteeism  measures  were  obtained  from 
Hackett  and  Guion's  (1985)  review  of  the  absenteeism 
literature  (.46  for  absence  frequency;  .66  for  time  lost 
indices) .  When  the  database  was  subgrouped  into  those 
using  frequency  based  measures  of  absenteeism  (22  samples 
with  a  total  sample  size  of  3,841)  and  those  using  time 
lost  measures  of  absenteeism  (9  samples  using  1,159  data 
points) ,  the  true  score  correlation  with  turnover  in  the 
two  subgroups  were  .34  and  .32,  respectively.  The  results 
of  the  subgroup  analysis  support  the  conclusion  from  the 
Bycio  (1992)  analyses  that  frequency  and  time  lost  measures 
have  similar  relationships  with  supervisory  ratings  of  job 
performance,  non-rating  quality  of  performance  measures, 
and  turnover. 

Hypothesized  Latent  Structures  of  Job  Performance 

The  earliest  conceptualizations  of  job  performance 
emphasized  the  economic  value  of  the  behaviors  of  an 
individual  to  the  organization  (i.e.,  the  economic 
construct  of  job  performance) .  The  value  of  behaviors,  not 
the  relations  between  the  behaviors  themselves  were 
stressed.  Given  the  restrictions  and  constraints  a  nascent 
field  faces  in  establishing  itself  as  a  useful  science 
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(Schmidt.  &  Kaplan,  1971)  such  an  emphasis  on  economic  value 
is  not  surprising.  For  example,  Toops  (1944)  identifies 
wages,  units  of  production,  absences  from  work,  quality  of 
work,  tenure,  supervisory  and  leadership  abilities  as  job 
performance  measures  and  discusses  how  these  measures 
affect  the  value  of  an  individual  to  the  organization. 
Bechtoldt  (1947,  p.357)  states  "a  criterion  is  a  means  of 
describing  the  performance  of  individuals  on  a  success 
continuum".  Brogden  and  Taylor  (1950)  explicitly  state, 
"the  criterion  should  measure  the  contribution  of  the 
individual  to  the  overall  efficiency  of  the  organization" . 
Horst  (1941,  p.20)  states  that,  "The  measure  of  success  or 
failure  in  an  activity  is  what  is  technically  known  as  the 
criterion" .  Finally,  in  the  same  vein,  Ghiselli  and  Brown 
(1948,  p.62)  state  that,  "By  criterion  is  meant  any 
attribute  or  accomplishment  of  the  worker  that  can  be  used 
as  an  index  of  his  [sic]  serviceability  or  usefulness  to 
the  organization  that  employees  him  [sic]".  As  I/O 
psychology  became  more  accepted  as  a  profession,  the  focus 
shifted  from  predicting  behaviors  of  economic  value  to  the 
organization  to  an  understanding  of  the  relations  between 
the  various  measures  of  job  performance. 

In  the  initial  years  of  examining  the  relationships 
between  the  job  performance  dimensions,  the  focus  was  on 
job  performance  measures  reflecting  the  productivity  of  the 
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individuals.  Over  time,  the  changing  demographic 
characteristics  of  the  workforce  and  the  increasing 
complexities  of  the  job  tasks  necessitated  more  job-  and 
organization-  specific  training.  This  lead  to  an  emphasis 
on  time  related  measures  of  job  performance  such  as 
absenteeism  and  turnover.  Also  the  emerging  literature  on 
expectancy  theory  as  well  as  the  increasing  difficulties 
(due  to  the  complexities  of  the  jobs)  in  evaluating 
behaviors  focused  the  attention  of  I/O  psychologists  on 
measures  of  effort  expenditure. 

Along  with  an  emphasis  on  time-related  measures  of  job 
performance,  there  was  a  focus  on  measures  of  job 
performance  reflecting  the  prosocial  behaviors  of  employees 
in  an  organization.  Organizational  citizenship  behavior  is 
defined  as  extra-role,  discretionary  behavior  that  helps 
other  organization  members  perform  their  jobs  (Bateman  & 
Organ,  1983;  Graham,  1986;  Smith,  Organ,  &  Near,  1983). 

This  builds  on  the  notion  of  the  informal  organization 
(Barnard,  1938)  and  spontaneous  behavior  (Katz  &  Kahn, 
1978).  Katz  (1964)  defines  prosocial  behavior  in  terms  of 
the  spontaneous,  co-operative,  helpful,  and  altruistic 
behaviors  chat  are  displayed  over  and  beyond  formal  role 
prescriptions.  The  literature  on  prosocial  and 
organizational  citizenship  behaviors  introduced  measures  of 
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job  performance  such  as  teamwork,  compliance,  altruism,  and 
helping  behaviors. 

Finally,  the  last  ten  years  have  seen  an  increase  in 
the  prediction  of  counterproductive  behaviors  at  work 
(Sackett,  Burris,  &  Callahan,  1989;  Sackett  &  Harris, 

1984).  There  has  been  an  increased  focus  on  measures  such 
as  theft  of  cash  or  merchandise  or  property,  damaging 
merchandise  to  buy  it  on  discount,  time  theft,  breaking 
rules,  preventable  accidents,  substance  abuse,  misuse  of 
discount  privileges,  vandalism,  physical  or  verbal  abuse  of 
peers,  supervisors,  or  customers,  and  so  on. 

As  could  be  expected,  studies  that  examine  the 
intercorrelations  between  all  (organizational  citizenship 
behaviors,  counterproductive  behaviors,  etc.,)  dimensions 
of  job  performance  is  necessarily  a  recent  phenomenon. 

Three  attempts  at  theoretical  integration  are  reviewed 
below.  The  first  by  Campbell  (1990b,  1990c)  examines  the 
relationships  between  the  various  dimensions  of  job 
performance  in  a  population  of  jobs.  The  second  by  Borman 
and  Motowidlo  (1992)  argues  specifically  for  the  inclusion 
of  contextual  performance  along  with  the  traditional 
measures  of  job  performance  in  theories  of  job  performance. 
Finally,  the  causal  model  of  job  performance  presented  by 
Schmidt  and  Hunter  (1992),  examining  the  relations  between 


the  construct  of  job  performance  and  their  causal 
determinants,  is  reviewed. 

Campbell  (1990b)  describes  the  latent  structure  of  job 
performance  in  terms  of  eight  general  factors.  His  eight 
factors  include  job  specific  taslc  proficiency,  non-job 
specific  tas>:  proficiency,  written  and  oral  communication 
skills,  demonstrating  effort,  maintaining  personal 
discipline,  facilitating  peer  and  team  performance, 
supervision,  and  management  or  administration. 

Job  specific  task  proficiency  is  defined  as  reflecting 
the  degree  to  which  the  individual  can  perform  the  core 
substantive  or  technical  tasks  that  are  central  to  a  job 
and  distinguish  one  job  from  another.  Non- job  specific 
proficiency,  on  the  other  hand,  is  used  to  refer  to  tasks 
not  specific  to  a  particular  job  but  expected  of  all 
members  of  the  organization.  Demonstrating  effort  captures 
the  consistency  or  perseverance  and  intensity  of  the 
individuals  to  complete  the  task,  whereas  maintenance  of 
personal  discipline  refers  to  the  eschewment  of  negative 
behaviors  (such  as  rule  infractions)  at  work.  Management/ 
administration  differs  from  supervision  in  that  the  former 
includes  performance  behaviors  directed  at  managing  the 
organization  that  are  distinct  from  supervisory  roles. 
Written  and  oral  communications  reflects  that  component  of 
job  performance  which  refers  to  the  proficiency  of  an 


incumbent  to  communicate  (written  or  oral)  independent  of 
the  correctness  of  the  subject  matter.  The  implications  of 
this  eight  factor  latent  structure  for  personnel  selection 
is  further  elaborated  in  Campbell  et  al.  (1992). 

According  to  Campbell  (1990b),  while  the  relative 
relevance  of  the  eight  factors  varies  across  jobs  (e.g., 
there  could  be  jobs  with  minimal  supervisory  performance) , 
the  eight  factors  or  some  subset  of  them  can  describe  the 
highest  order  latent  variables  for  every  job  in  the 
occupational  domain.  Further,  he  asserts  that  three 
factors--core  taslc  proficiency,  demonstrated  effort,  and 
maintenance  of  personal  discipline--are  constituents  of  job 
performance  in  every  job.  Finally,  Campbell  (1990b)  in 
addressing  the  nature  of  lower  order  factors  points  out  the 
lack  of  any  theoretical  guideline  and  states  that  the 
description  of  the  lower  order  factors  remains  a  matter  of 
speculation  (at  one  extreme  we  can  hypothesize  as  many 
latent  structures  of  job  performance  as  there  are  jobs) . 

Borman  and  Motowidlo  (1992)  argue  for  expanding  the 
criterion  domain  to  include  elements  of  contextual 
performance  in  addition  to  the  traditional  measures  of  task 
(or  job  specific)  performance  Drawing  from  the  literature 
on  organizational  citizenship  behavior  (Bateman  &  Organ, 
1983;  Smith  et  al.,  1983),  prosocial  organizational 
behavior  (Brief  &  Motowidlo,  1986;  Graham,  1986;  Katz  & 
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Kahn,  1978;  Organ,  1988),  and  a  model  of  soldier 
effectiveness  developed  in  Project  A  (Campbell,  1990c) , 
Borman  and  Motowidlo  (1992)  attempt  to  define  the  elements 
of  contextual  performance.  According  to  Borman  and 
Motowidlo  (1992),  contextual  performance  cuts  across 
different  jobs  (organization-wide,  not  job  specific)  in  an 
organization  and  includes  performance  components  such  as: 

(a)  persisting  with  enthusiasm  and  extra  effort  as 
necessary  to  complete  own  taslc  activities  successfully,  (b) 
volunteering  to  carry  out  tas)<  activities  that  are  not 
formally  part  of  own  job,  (c)  helping  and  cooperating  with 
others,  (d)  following  organizational  rules  and  procedures, 
and  (e)  endorsing,  supporting  and  defending  organizational 
objectives . 

Finally,  the  causal  model  of  job  performance  developed 
by  Schmidt  and  Hunter  (1992)  views  conscientiousness  and 
general  mental  ability  as  common  causal  antecedents  of 
effort,  citizenship  behaviors,  job  performance,  skill 
acquisition  and  performance  capability.  For  example, 
general  mental  ability  is  hypothesized  to  have  an  effect  on 
job  skill  acquisition  and  conscientious  employees  are 
hypothesized  to  spend  more  time  on  their  jobs,  become  more 
skilled,  and  engage  more  in  helping  other  employees.  Thus, 
two  common  causal  antecedents  (General  Mental  Ability  and 
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Conscientiousness)  are  expected  to  predict  a  wide  variety 
of  job  performance  measures. 

Problems  and  Limitations 

The  literature  on  the  interrelationships  among  the 
various  dimensions  of  job  performance  has  evolved  around 
four  screams  of  research.  First,  based  on  a  synthesis  of 
the  available  knowledge  scientists  have  focused  on 
specifying  the  domain  of  the  construct  of  job  performance. 
Essentially,  this  line  of  research  has  focused  on 
identifying  and  specifying  the  meaning  of  words  associated 
with  the  construct  of  job  performance.  The  second  stream 
of  research  has  examined  the  interrelationships  between  the 
various  dimensions  of  job  performance  based  on  information 
collected  regarding  task,  performed  on  the  jobs.  Task 
surveys  (of  incumbents,  supervisors,  or  subordinates)  of 
ti^he  time  spent  on  and  importance  of  various  tasks,  content 
analysis  of  performance  feedback  episodes,  use  of  critical 
incident  techniques,  and  (unobtrusive)  observation  of  job 
performance  are  some  techniques  used  to  collect  information 
regarding  the  tasks  performed  on  the  jobs.  The  data 
obtained  is  usually  analyzed  through  factor  analytic 
techniques  or  delphi  processes  to  explicate  the 
relationships  between  the  various  dimensions  of  job 
performance.  The  third  line  of  research  found  in  the 
literature  on  the  relationships  between  the  various 


dimensions  of  job  performance  focuses  on  obtaining  scores 
for  a  sample  of  individuals  on  some  measures  of  job 
performance.  An  analysis  of  the  intercorrelations  between 
these  measures  is  used  to  examine  the  factors  that  underlie 
the  measures  included  in  the  analysis.  The  final  strand  of 
research  in  the  literature  relevant  to  the  investigation  of 
the  correlations  between  the  various  dimensions  of  job 
performance  are  studies  that  employ  meta-analytic 
techniques  to  cumulate  results  across  studies  reporting 
correlations  between  a  pair  of  measures  of  job  performance. 
These  studies  address  the  convergent  validity  of  the  two 
measures  of  job  performance,  the  correlation  between  which 
is  being  cumulated  across  studies. 

The  relevance  and  value  of  the  four  streams  of  research 
in  investigating  the  relationships  between  the  various 
dimensions  of  job  performance  can  be  evaluated  in  terms  of 
the  comprehensiveness  of  the  analysis,  the  empirical 
verifiability  of  the  hypothesized  relationships,  the 
cumulativeness  of  the  empirical  evidence  available  to  date, 
and  the  extent  to  which  statistical  artifacts  are  addressed 
before  inferences  are  drawn.  Empirical  verification  is 
required  to  test  the  hypothesized  relationships  between  the 
set  of  measures.  The  approach  used  to  explicate  the 
relationships  between  the  various  dimensions  of  job 
performance  must  also  cumulate  results  across  studies 
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examining  the  correlation  between  the  same  pair  of  job 
performance  dimensions.  The  effects  of  numerous 
statistical  artifacts  should  be  addressed  and  the  results 
across  studies  cumulated  before  firm  conclusions  can  be 
drawn  about  the  relationships  between  the  various 
dimensions  of  job  performance. 

A  comprehensive  analysis  of  the  psychological  construct 
of  job  performance  must  include  all  measures  of  job 
performance  (that  have  been  employed  in  the  literature  over 
the  years  and  for  which  data  is  available) .  A  central 
thesis  in  this  dissertation  is  that  none  of  the  previous 
attempts  to  explicate  the  psychological  construct  of  job 
performance  has  been  comprehensive.  A  comprehensive 
specification  of  the  psychological  construct  of  job 
performance  (encompassing  all  potential  measures  of  job 
performance)  can  be  obtained  by  compiling  a  list  of  the 
different  measures  of  job  performance  used  in  the 
individual  studies.  Parallel  to  the  lexical  hypothesis  of 
Galton  (Goldberg,  1992)  which  states  that  a  complete 
description  of  the  personality  of  an  individual  can  be 
found  within  the  list  of  words  in  the  dictionary  (i.e.,  the 
adjectives  used  in  the  language) ,  I  hypothesize  that  the 
psychological  construct  of  job  performance  can  be  captured 
by  analyzing  the  various  measures  of  job  performance  used 
in  the  many  individual  studies  over  the  years.  That  is, 


each  individual  study  focuses  on  a  subset  of  all  possible 
manifestations  of  the  underlying  construct  of  job 
performance,  but  by  cumulating  across  all  individual 
studies  it  is  possible  to  cover  all  manifestations  of  the 
underlying  construct  of  job  performance. 

Though  rational  analysis  can  encompass  the  whole  domain 
of  job  performance,  in  actual  practice  rational  analysis 
has  been  confined  to  the  idiosyncrasies  of  the  individual 
researcher  synthesizing  the  available  knowledge.  The 
Zeitgeist  (the  spirit  of  the  times)  and  the  ortgeist  (the 
spirit  of  the  place)  decermine  to  a  great  extent  what 
measures  are  considered  in  the  synthesis.  Further,  the 
speculative  nature  of  rational  analysis  makes  it  impossible 
to  compare  quantitatively  the  various  models  of  the 
psychological  construct  of  job  performance  based  on 
rational  analysis.  As  such  cumulation  or  integration  of 
evidence  based  on  rational  analysis  has  to  be  largely 
subj  ective . 

Individual  factor  analytic  studies  correlating  a  subset 
of  the  various  potential  measures  of  job  performance 
facilitate  empirical  verification  of  the  hypothesized 
relationships;  but  they  are  not  capable  of  resolving  the 
relationships  of  interest  because  of  the  numerous 
statistical  artifacts  affecting  the  individual  studies. 
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Further  they  do  not  provide  a  comprehensive  coverage  of  the 
psychological  construct  of  job  performance. 

Finally,  the  use  of  meta-analysis  has  been  very  limited 
in  the  literature.  To  date,  the  convergent  validity  of 
only  six  pairs  of  measures  (supervisory  ratings  of  job 
performance-production  records;  absenteeism-supervisory 
ratings  of  30b  performance;  absenteeism- turnover ; 
absenteeism-production  records;  absenteeism-organizational 
records  of  quality  of  performance;  peer-supervisory  ratings 
of  job  performance)  have  been  examined  using  meta-analysis 
(Bycio,  1992;  Harris  &  Schaubroeck,  1988;  Heneman,  1986; 
McEvoy  Sc  Cascio,  1987;  Mitra  et  al.,  1992).  Further, 
establishing  the  true  score  correlations  between  measures 
is  only  the  first  step  in  testing  the  psychological 
construct  of  job  performance.  The  true  score  correlations 
should  be  used  as  building  blocks  with  confirmatory  factor 
analysis  and  latent  variable  analysis  to  explicate  the 
psychological  construct  of  job  performance.  What  is  needed 
IS  a  research  project  that  comprehensively  (i.e.,  includes 
all  measures  that  have  been  used  so  far  in  the  literature) 
specifies  the  psychological  construct  of  job  performance, 
empirically  tests  the  hypothesized  relationships  between 
the  various  measures,  removes  the  effects  of  statistical 
artifacts,  cumulates  research  findings  across  individual 
studies,  and  through  the  use  of  confirmatory  factor 


analysis  and  latent  variable  analysis  infers  conclusions 
regarding  the  psychological  construct  of  job  performance. 

Summary  Outline  of  Dissertation 

Despite  the  problems  and  limitations,  the  dimensions  of 
job  performance  found  in  the  literature  revolve  around  five 
themes.  First,  there  are  job  performance  measures  that 
focus  on  productivity.  Number  of  units  produced,  number  of 
transactions  completed  are  some  examples  of  job  performance 
measures  that  reflect  productivity.  Second,  there  are  job 
performance  measures  chat  reflect  how  conscientious 
individuals  are.  The  effort  expended  on  the  job,  the 
quality  of  production/work,  compliance  with  rules,  not 
being  involved  in  accidents  reflect  the  same  theme  of  how 
conscientious  and  meticulous  the  individual  is.  Third, 
there  are  job  performance  measures  that  assess  how  well  an 
individual  interacts  with  others.  Interpersonal  relations 
with  others,  leadership,  administrative  skills  are  some 
dimensions  of  job  performance  that  reflect  how  well  an 
individual  interacts  with  others.  A  fourth  group  of  job 
performance  measures  captures  time  related  indices  such  as 
absenteeism  and  turnover.  Finally,  a  fifth  group  of 
measures  focuses  on  overall  evaluation  of  the  individuals 
(i.e.,  global  measures  of  performance).  I  will  use  these 
five  groups  to  define  the  group  factors  in  the  second  level 
of  a  three  level  hierarchical  model .  The  important  point 


41 


is  not  whether  the  group  factors  identified  are  absolute; 
alternate  groups  of  factors  can  be  postulated  and  alternate 
three  level  hierarchical  models  can  (and  should)  be  tested 
using  the  true  score  correlations  reported  here.  (One 
could  hypothesize  different  predictors  for  the  different 
group  factors;  conscientiousness  will  be  the  best  predictor 
of  effort,  agreeableness  will  be  the  best  predictor  of 
interpersonal  relations,  etc.) 

In  this  dissertation,  I  cumulate  the  results  across 
individual  studies  reporting  correlations  between  various 
measures  of  job  performance.  The  first  step  is  obtaining 
studies  that  measures  performance  of  individual  employees 
on  more  than  one  measure  of  job  performance.  The  second 
step  is  identifying  the  various  conceptually  distinct 
measures  of  job  performance  that  have  been  used  in  the 
literature.  The  third  step  is  establishing  the  true  score 
correlations  between  the  identified  measures.  The  fourth 
and  final  step  is  testing  whether  an  hypothesized 
hierarchical  latent  structure  model  fits  the  pattern  of 
true  score  correlations  between  the  various  measures  of  job 
performance.  I  test  both  a  two-  and  a  three-  level 
hierarchical  model.  In  the  next  chapter,  the  methods  used 
are  presented. 
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CHAPTER  III 

METHOD 

Database  Description 

Inclusion  Criteria 

To  estimate  the  true  score  correlations  between  the 
different  dimensions  of  job  performance  used  in  the 
literature,  I  searched  the  literature  in  industrial/ 
organizational  psychology,  organizational  behavior,  and 
human  resources  management.  Studies  that  report  the 
correlations  between  the  different  dimensions  of  job 
performance  were  obtained.  To  be  included  in  the  database, 
a  study  had  to  be  from  a  published  source.  Conference 
presentations  and  unpublished  data  were  not  included. 

The  exclusion  of  unpublished  studies  in  this 
dissertation  is  only  for  feasibility  of  data  management. 

It  is  not  possible  to  identify  studies  that  report 
correlations  between  various  measures  of  job  performance 
without  obtaining  a  copy  of  the  full  study.  For  example,  a 
published  study  comparing  grievants  with  non-grievants 
(e.g.,  Sulkin  &  Pranis,  1967)  in  an  organization  reported 
the  correlation  between  absenteeism  and  tenure.  Neither 
the  title  nor  the  abstract  of  this  published  study  (Sulkin 
&  Pranis,  1967)  indicates  that  a  correlation  that  could  be 
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of  use  for  this  dissertation  is  reported;  only  after 
scanning  through  the  entire  article  was  such  a  correlation 
identified  and  the  study  was  included  in  the  database.  A 
similar  search  strategy  is  impossible  with  all  the 
dissertations  and  unpublished  studies  done  in  I/O 
Psychology.  Therefore,  I  did  not  include  dissertations, 
books,  and  unpublished  reports  in  this  research. 

This  exclusion  of  unpublished  studies  is  not  an 
endorsement  of  misleading  and  erroneous  arguments  calling 
for  the  exclusion  of  unpublished  studies  in  all  meta¬ 
analyses  on  the  grounds  that  such  unpublished  studies 
constitute  poor  quality  data.  (The  converse  argument 
maintains  that  published  studies  have  a  positive  bias  that 
overstate  the  results.  Taken  together,  these  two  arguments 
will  lead  to  scientific  nihilism  [Hunter  &  Schmidt,  1990a, 
p.515].)  The  hypothesis  of  methodological  inadequacy  of 
unpublished  studies  (in  comparison  to  published  studies) 
has  not  been  established  in  any  research  area.  In  fact, 
ample  evidence  exists  to  prove  the  comparability  of 
findings  of  published  and  unpublished  studies  in  many 
research  areas  (Hunter  &  Schmidt,  1990a,  pp.  507-509). 

Hunter  and  Schmidt  (1990a,  pp.  509-510)  present  a 
hypothetical  example  that  illustrates  how  differences 
between  published  and  unpublished  studies  examining  the 
effectiveness  of  psychotherapy  could  have  been  due  to 
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statistical  artifacts.  Ones  et  al.  (in  press)  found  that 
the  correlation  between  the  reported  validity  of  integrity 
tests  and  the  dichotomous  variable  indicating  published 
versus  unpublished  studies  is  negligible.  In  the 
literature  on  the  validity  of  employment  tests,  impressive 
evidence  has  been  accumulated  which  indicates  that 
published  and  unpublished  studies  do  not  differ  in  the 
validities  reported  (Hunter  &  Schmidt,  1990a,  pp.  507-509). 
For  example,  the  data  used  by  Pearlman,  Schmidt,  and  Hunter 
(1980)  was  found  to  be  very  similar  to  the  U.S.  Department 
of  Labor  (GATE)  data  base  used  by  Hunter  (1983)  and  other 
large  sample  military  data  sets.  Also  the  mean  validities 
in  the  Pearlman  et  al.  (1980)  data  base  are  virtually 
identical  to  Ghiselli’s  (1966)  reported  medians.  Further, 
the  percent  of  nonsignificant  studies  in  the  Pearlman  et 
al.  (1980)  data  base  perfectly  matches  the  percent  of 
nonsignificant  published  studies  reported  by  Lent, 

Auerbach,  and  Levin  (1971) .  Finally,  the  percentage  of 
observed  validities  that  were  nonsignificant  at  the  .05 
level  in  the  Pearlman  et  al.  (1980)  data  base  (56.1%  of  the 
2,795  observed  validities)  is  consistent  with  the  estimate 
obtained  by  Schmidt,  Hunter,  and  Urry  (1976),  that  the 
average  criterion-related  validation  study  has  statistical 
power  no  greater  than  .50.  If  selectivity  or  bias  in 
reporting  were  operating  many  of  the  nonsignificant 
validities  would  have  been  omitted,  and  the  percent 


45 


significanc  should  have  been  higher  than  43.9%.  On  the 
other  hand,  if  unpublished  studies  were  of  poorer  quality, 
not  meeting  the  standards  of  peer  review,  then  there  should 
have  been  more  non-significant  validities  among  the 
unpublished  studies  than  56%.  Thus,  there  is  ample 
evidence  arguing  for  the  equivalence  of  published  and 
unpublished  studies.  The  two  data  bases  are  often 
comparable.  The  sole  reason  for  confining  this 
dissertation  to  published  studies  is  feasibility  of  data 
mar.igement . 

In  searching  the  literature  I  employed  both  electronic 
and  manual  search  strategies.  Psyclit  database  was 
searched  for  the  period  1974-1991,  and  the  keywords  used  in 
the  search  are  provided  in  Appendix  A.  The  same  search  was 
carried  out  in  Wils,  Infotrac,  Oasis,  and  CLRC  databases. 
However,  the  most  fruitful  search  strategy  was  the  manual 
search  strategy.  I  searched  the  major  journals,  volume  by 
volume,  issue  by  issue,  for  articles  that  report  the 
correlation  between  any  two  dimensions  of  job  performance. 
The  references  of  these  articles  were  also  searched  for 
identifying  further  relevant  articles.  In  short,  a 
snowballing  technique,  with  the  articles  published  in  the 
major  journals  as  the  initial  kernel,  was  the  most 
effective  strategy  in  identifying  the  articles  used  in  this 
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dissertation.  The  journals,  all  issues  of  which  were 
searched,  are  listed  in  Appendix  B. 

I  included  in  the  database  the  studies  that  report  data 
on  individuals  regarding  their  performance  on  actual  jobs. 
Studies  reporting  data  on  departments  or  production  units 
were  not  included.  Experimental  simulations  reporting  data 
on  student  samples  engaged  in  laboratory  work  {or  ratings 
of  videotaped  and  manipulated  work  samples)  were  excluded. 
Assessment  center  ratings  as  well  as  interviewer  and 
recruiter  ratings  were  omitted  because  they  reflect  scores 
on  predictors  rather  than  job  performance. 

Self-reports,  observer's  reports,  subordinate  ratings, 
and  customer  ratings  of  performance  were  excluded.  It  is 
not  enough  to  observe  or  record  behavior;  we  must  evaluate 
it  (must  know  what  the  behavior  means) .  Observing  that 
subjects  do  a  specific  task  does  not  reveal  the  hundreds  of 
subtle  differences  in  how  it  is  done  that  makes  one 
successful  and  another  not  (Schmidt,  1992).  Further,  the 
evaluator  should  know  the  importance  or  consequence  of  a 
particular  behavior  to  the  job.  The  lack  of  this  expertise 
in  evaluation  is  the  reason  for  excluding  self-reports, 
observers'  reports,  subordinate  ratings,  and  customer 
ratings  of  performance  in  this  dissertation. 
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When  studies  reported  only  the  range  (or  average)  of 
the  intercorrelations  between  the  measures  such 
intercorrelations  could  not  be  classified  as  representing 
the  intercorrelation  between  a  particular  pair  of  job 
performance  dimensions.  As  such  these  studies  were 
excluded  from  the  database.  Finally,  the  database  also 
includes  studies  that  report  only  the  interrater 
reliability  and  internal  consistency  estimates  (coefficient 
alphas)  for  the  different  dimensions  of  job  performance. 

The  reliability  values  were  compiled  to  construct  artifact 
distributions  (elaborated  below)  that  were  used  in  the 
psychometric  meta-analyses. 

The  same  dimension  of  job  performance  can  be  referred 
to  by  different  labels.  In  cumulating  results  across 
studies  a  dimension  of  job  performance  could  be  referred  to 
by  different  labels  in  the  different  studies.  The  taslc  of 
identifying  the  various  dimensions  of  job  performance  and 
eliminating  redundancies  has  to  be  guided  by  conceptual 
considerations.  At  one  extreme  even  the  changing  of  a 
single  letter  in  the  definition  of  the  dimension  can  be 
construed  as  signifying  a  new  and  distinct  dimension  of  job 
performance.  Up  to  a  point,  it  can  be  proposed  that  the 
more  fine-grained,  narrow,  and  explicitly  defined  the 
dimensions  of  job  performance  are,  the  greater  the 
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conceptual  clarity  and  interpretability  of  empirical 
results. 

Any  advantage  in  clarity  and  interpretability  of 
empirical  results  that  narrowly  defined  dimensions  have 
over  more  broadly  defined  dimension's  is,  however,  offset  by 
considerations  of  availability  of  data  and  robustness  of 
resulting  estimates.  When  results  are  cumulated  across 
studies,  intercorrelations  between  some  narrowly  defined 
dimensions  may  not  be  available,  necessitating  the  analysis 
at  a  level  where  the  various  dimensions  of  job  performance 
are  defined  more  broadly  (i.e.,  necessitating  combining  of 
similar  dimensions) .  Further,  if  the  various  dimensions  of 
job  performance  are  narrowly  defined,  the  number  of 
correlations  for  establishing  (using  the  methods  of 
psychometric  meta-analysis)  the  true  score  intercorrelation 
between  two  dimensions  will  be  small.  The  subsequent  small 
total  sample  sizes  included  in  the  psychometric  meta- 
analysis  will  lead  to  substantial  first  order  sampling 
error.  Also  the  small  number  of  correlations  used  in 
psychometric  meta-analysis  will  increase  the  second  order 
sampling  error.  Though  the  estimated  mean  true  score 
intercorrelation  may  not  be  greatly  affected  by  the  second 
order  sampling  error,  the  true  variability  will  often  be 
distorted  (either  under  or  overestimated) . 
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Further,  the  hypothesized  high  conceptual  clarity  that 
accrues  from  more  narrowly  defined  dimensions  should  also 
be  balanced  against  the  generality  of  the  dimensions.  For 
example,  suppose  we  define  three  measures  as:  (a) 
supervisory  ratings  of  the  frequency  with  which  an  employee 
smiles  at  his  or  her  peers;  (b)  smiles  at  supervisors;  and 
(c)  smiles  at  customers.  Now,  suppose  we  include  these 
three  measures  with  other  measures  of  job  performance  such 
as  productivity,  leadership,  dependability,  etc.  The  three 
measures  involving  smiling  will  probably  have  a  pattern  of 
correlations  that  sets  them  apart  from  other  measures, 
causing  the  appearance  of  a  smilability  factor.  However, 
the  question  remains  whether  such  narrowly  defined 
performance  measures  are  important  to  anv  job.  Even  if 
there  is  a  particular  iob  where  frequency  of  smiling  is 
important,  it  is  highly  unlikely  that  I/O  psychologists 
would  consider  the  inclusion  of  such  narrowly  defined 
measures  as  worth  the  labor  involved  in  constructing 
prediction  instruments  and  theories  of  job  performance 
tailored  for  a  particular  job.  Developing  theories  of  job 
performance  for  each  task  (or  even  job)  will  hinder  the 
development  of  a  general  theoretical  understanding  of  the 
construct  of  job  performance.  As  the  content  generality  of 
the  dimensions  increase,  the  value  of  the  dimensions  in 
developing  prediction  instruments  and  theories  of  work 


behavior  increases. 
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However,  the  identified  dimensions  of  job  performance 
should  be  conceptually  distinct  and  completely  exhaustive. 
Conceptual  distinctiveness  requires  that  any  specific 
measure  used  in  a  study  should  be  unambiguously  identified 
as  referring  to  a  particular  dimension  of  job  performance. 

A  completely  exhaustive  scheme  of  job  performance  measures 
ensures  that  all  dimensions  of  performance  in  any  job  are 
covered. 

The  various  dimensions  of  job  performance  analyzed  in 
this  dissertation  were  identified  by  grouping  the  list  of 
measures  obtained  from  the  studies  into  conceptually 
similar  groups.  The  identified  dimensions  of  job 
performance  were  cross -validated  by  having  another 
researcher  (Deniz  Ones)  independently  complete  the  taslc  of 
grouping  the  list  of  measures  obtained  from  the  studies 
into  conceptually  similar  groups.  Differences  (overall 
agreement  rate  was  92.4%)  were  resolved  through  mutual 
discussions  until  consensus  was  reached.  A  list  of  the 
various  dimensions  identified  and  used  in  this  dissertation 
is  provided  in  Appendix  C. 

Some  studies  reported  correlations  between  different 
wordings  of  the  same  dimensions  along  with  correlations 
with  other  dimensions  of  job  performance.  A  composite 
measure  was  formed  over  the  different  wordings  of  the  same 
dimension.  The  correlation  of  this  composite  measure  with 


other  dimensions  of  job  performance  was  computed.  These 
computed  correlations  were  used  in  the  meta-analyses. 
Similarly,  the  reliability  of  the  composite  of  the 
different  wordings  of  the  same  dimension  was  computed  using 
the  Hosier  formula  and  recorded.  Unit  (equal)  weights  for 
the  different  wordings  of  the  same  dimension  were  used  in 
computing  the  composite.  An  alternative  type  of 
reliability  estimate  for  the  composite  is  the  estimate 
based  on  the  standardized  coefficient  alpha,  which  is  the 
same  as  the  use  of  the  Spearman-Brown  formula  based  on  the 
average  intercorrelation  between  the  different  wordings  of 
the  same  dimension.  However,  the  use  of  either  of  these 
would  assign  the  specific  factor  variance  to  measurement 
error.  To  examine  whether  there  is  a  psychological 
construct  underlying  the  various  dimensions,  it  is 
appropriate  to  assign  the  specific  factor  variance  to 
measurement  error.  The  use  of  Hosier  formula-  (instead  of 
Spearman- Brown  or  the  standardized  coefficient  alpha) 
results  in  a  higher  estimate  of  the  reliability  (because 
the  specific  factor  variance  is  treated  as  true  variance) , 
which  lowers  the  corrected  correlation  and  provides  a  more 
rigorous  test  of  the  hypothesis  that  all  measures  of  job 
performance  are  manifestations  of  the  same  construct.  That 
IS,  if  the  correlations  corrected  with  Hosier  reliability 
support  the  existence  of  a  general  factor,  the  use  of  the 
Spearman-Brown  reliability  (or  standardized  coefficient 


alpha)  in  which  the  specific  factor  variance  is  removed 
from  the  true  variance  will  result  in  the  emergence  of  a 
larger  general  factor. 
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Data  Coded/Extracted  from  Primary  Studies 

An  identification  number  was  given  to  each  study.  The 
list  of  studies  coded  is  provided  in  Appendix  D.  When  more 
than  one  sample  was  reported  in  a  study,  a  sample  within 
study  identification  number  was  given  to  each  sample  within 
that  study.  Samples  were  numbered  consecutively  starting 
with  the  numiber  one.  If  a  study  reports  data  on  a  total 
sample  as  well  as  on  subsamples  (say  blacks  and  whites,  or 
males  and  females),  I  included  the  data  only  from  the  total 
sample.  Subgroup  correlations  are  prone  to  capitalization 
on  chance  and  averaging  subgroup  correlations  will 
introduce  a  downward  bias  in  the  correlation  (due  to  range 
rescriction)  as  compared  to  the  correlation  computed  on  the 
total  sample  (Hunter  &  Schmidt,  1990a,  p.465). 

Each  sample  in  a  study  typically  reports  more  than  one 
intercorrelation  (e.g.,  if  the  sample  in  a  study  examines 
four  dimensions  of  job  performance,  there  will  be  six 
intercorrelations) .  A  separate  record  was  created  for  each 
intercorrelation.  An  identification  number  was  also  given 
to  each  dimension  of  job  performance  (the  same  number  is 
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also  referenced  in  Appendix  C  where  the  dimensions  of  job 
performance  are  listed) . 

Each  record  includes  a  study  number,  a  sample  number, 
and  the  observed  correlation  between  two  dimensions  of  job 
performance.  Each  record  also  contains,  for  each  of  the 
two  dimensions  of  job  performance  (the  correlation  between 
which  is  reported  in  that  record) ,  an  identification  number 
for  the  dimension  of  job  performance  (the  same  number  as 
reported  in  Appendix  C),  the  interrater  reliability  of  that 
dimension,  the  internal  consistency  of  the  dimension,  and 
the  degree  of  split  (if  any)  in  that  dimension  of  job 
performance.  If  reliability  information  is  not  available, 
the  columns  in  that  record  were  1-ft  blanlc.  If  a  record 
involves  a  study  that  reports  only  reliability  information 
for  a  particular  dimension  of  job  performance,  the  columns 
for  the  observed  correlation,  sample  size,  degree  of  split 
as  well  as  the  columns  for  the  second  dimension  of  job 
performance  were  left  blan)<.  (The  sample  size  and  the 
reliab; lity  of  dimensions  were  used  to  assign  appropriate 
weights  to  the  study  results.  The  degree  of  split  was  used 
to  correct  the  artificially  dichotomized  correlations.) 

In  coding  the  interrater  reliabilities,  care  was  taJcen 
to  adjust  the  reported  values  using  Spearman-Brown 
corrections  wherever  appropriate.  For  example,  whenever  a 
study  reported  the  interrater  reliability  based  on  one 
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rater  as  .50  but  the  correlation  between  the  various 
dimensions  of  job  performance  was  based  on  the  sumjned  or 
average  ratings  of  six  raters,  the  reliability  was  coded  as 

.86. 


Intercoder  agreement  in  summarizing  or  extracting 
information  from  the  primary  studies  is  a  concern  in  meta¬ 
analyses.  Haring  et  al.  (1981)  present  empirical  data 
indicating  that  intercoder  agreement  in  meta-analyses  is  a 
function  of  the  judgmental  nature  of  the  items  coded.  The 
Haring  et  al.  (1981)  review  of  meta-analyses  found  that 
eight  of  the  nine  items  lowest  in  coder  agreement  were 
judgments  (e.g.,  the  quality  of  the  study)  as  opposed  to 
calculation  based  variables  (e.g.,  effect  sizes,  number  of 
subjects).  JacJcson  (1980)  and  Hattie  and  Hansford  (1982, 
1984)  also  provide  data  which  indicate  that  problems  of 
intercoder  agreement  in  meta-analyses  are  negligible  for 
coding  computation-based  numerical  variables.  Finally, 
Whetzel  and  McDaniel  (1988)  found  no  evidence  of  any  coder 
disagreements  in  validity  generalization  data  bases. 
Intercoder  agreement  in  this  dissertation  was  estimated  by 
having  another  individual  (Deniz  Ones)  code  a  subset  of 
studies.  The  percentage  agreement  between  us  in  coding  the 
selected  subset  of  studies  was  97,6%. 
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Psychometric  Meta-Analvses 

Data  from  the  sources  described  in  the  previous  section 
was  cumulated  by  the  methods  of  psychometric  meta-analyses. 
Depending  on  the  availability  of  information  in  the  primary 
studies,  we  can  either  correct  the  observed  correlations 
for  the  effects  of  statistical  artifacts  and  cumulate  the 
individually  corrected  correlations,  or  use  artifact 
distributions  to  correct  the  observed  distribution  of 
correlations,  or  use  a  combination  of  individual 
corrections  and  artifact  distributions. 

Because  the  degree  of  split  for  dichotomization  is  • 
usually  given  in  the  research  reports,  it  was  possible  to 
correct  the  correlations  individually  for  the  attenuating 
effects  of  dichotomization.  But  to  correct  for  the  effects 
of  artifacts  such  as  unreliability,  where  the  information 
available  is  sporadic,  recourse  was  made  to  the  use  of 
artifact  distributions.  That  is,  a  mixed  meta-analysis  was 
employed,  in  the  first  step,  the  correlations  were 
corrected  individually  for  the  effects  of  dichotomization. 
In  the  second  step,  the  partially  corrected  distribution 
obtained  from  the  first  step  was  corrected  for 
unreliability  in  the  measures  of  the  two  variables  being 
correlated  using  artifact  distributions  (Hunter  &  Schmidt, 
1990a,  p.188) . 


Artificially  dichotomized  correlations  were  corrected 
for  discontinuity  (Hunter  &  Schmidt,  1990b) .  In  correcting 
for  dichotomization,  care  was  taken  to  account  for  the 
conceptualization  of  the  measure  (Williams,  1990).  That 
is,  corrections  for  dichotomization  were  undertaken  only 
where  it  is  theoretically  defensible.  For  example,  if  an 
absenteeism  measure  was  dichotomized  (as  high  vs.  low)  then 
dichotomization  corrections  were  applied.  The  correlations 
were  corrected  individually.  When  dichotomization 
corrections  were  applied  to  the  observed  correlations,  the 
sample  sizes  were  adjusted  to  estimate  the  correct  sampling 
error  variance  (Hunter  &  Schmidt,  1990b) .  First,  the 
uncorrected  correlation  and  the  study  sample  size  were  used 
to  estimate  the  sampling  error  variance  for  the  observed 
correlation.  This  value  was  then  multiplied  by  the  square 
of  the  dichotomization  factor  (the  ratio  of  the  corrected 
to  uncorrected  correlation) ,  yielding  the  sampling  error 
variance  associated  with  the  dichotomization  corrected 
correlation.  This  value  was  then  used  with  the  uncorrected 
correlation  in  the  standard  sampling  error  formula  to  solve 
for  the  adjusted  sample  size  used  in  the  meta-analyses. 
Entry  of  this  sample  size  into  the  meta-analysis 
calculations  results  in  the  correct  estimate  of  the 
sampling  error  variance  of  the  corrected  correlation  in  the 
meta-analysis. 
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After  the  correlations  were  corrected  individually  for 
dichotomization,  artifact  distribution  meta-analysis  was 
used  to  correct  for  unreliability  in  the  measures.  In 
using  artifact  distributions  for  correcting  two  or  more 
artifacts  we  have  the  option  to  use  either  the  interactive 
procedure  which  corrects  the  observed  correlations  for  the 
effects  of  the  various  statistical  artifacts 
simultaneously,  or  the  noninteractive  procedure  which 
corrects  the  observed  correlation  for  the  effects  of  the 
statistical  artifacts  sequentially  (one  after  another) . 
Recent  computer  simulation  studies  (e.g..  Law,  Schmidt,  & 
Hunter,  1992;  Schmidt  et  al . ,  1993)  have  shown  that  among 
the  methods  of  psychometric  meta-analyses  the  interactive 
procedure  used  with  certain  refinements,  such  as  nonlinear 
range  restriction  and  mean  observed  correlation  in  the 
sampling  error  formula,  is  the  most  accurate  one.  As  such, 
the  Hunter -Schmidt  interactive  meta-analytic  procedure  was 
used. 

In  correcting  for  unreliability  in  the  measures,  the 
use  of  the  correct  form  of  reliability  coefficient  requires 
the  specification  of  the  nature  of  the  error  of  measurement 
in  the  research  domain  of  interest  (Hunter  &  Schmidt, 

1990a,  pp.  123-125) .  The  two  measures  being  correlated 
can:  (a)  both  be  organizational  records;  or  (b)  both  be 

ratings  provided  by  the  same  individual;  or  (c)  both  be 
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ratings,  but  the  two  ratings  are  provided  by  two  different 
individuals;  or  (d)  one  measure  is  a  rating  while  the  other 
is  an  organizational  record  based  measure.  When  the 
correlation  is  between  two  organizational  records,  random 
error  in  responses  was  corrected  by  using  an  artifact 
distribution  of  coefficient  alphas  and  test-retest 
reliabilities.  To  correct  the  correlation  between  two 
ratings  when  both  the  ratings  are  provided  by  the  same 
individual  (intra  individual  correlations),  intra  rater 
reliability  estimates  were  used.  Because  both  ratings  are 
provided  by  the  same  individual,  use  of  interrater 
reliability  coefficients  is  inappropriate  (interrater 
disagreements  are  not  involved  in  such  correlations) .  Halo 
is  not  corrected  for  when  the  true  score  correlations 
between  two  ratings  (when  both  the  ratings  are  provided  by 
the  same  individual--intra  individual  correlations)  are 
reported. 

When  one  of  the  two  ratings  being  correlated  is 
provided  by  supervisors  whereas  the  other  is  provided  by 
peers,  the  observed  correlations  are  free  of  halo. 

However,  we  need  to  correct  the  observed  correlation  for 
interrater  disagreements.  Therefore,  when  the  two  ratings 
being  correlated  came  from  two  different  individuals,  I 
used  the  interrater  reliability  estimates  in  the 


corrections . 
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A  separate  distribution  of  reliabilities  for  each 
dimension  of  job  performance  was  constructed  from  the  data 
available  in  the  literature.  If  no  study  was  available 
that  reported  the  reliability  of  some  dimension  of  job 
performance,  then  for  that  dimension  of  job  performance,  an 
appropriate  distribution  of  reliabilities  was  constructed 
using  the  reliability  of  other  similar  dimensions  of  job 
performance . 

The  result  of  a  psychometric  meta-analysis  is  an 
estimate  of  the  distribution  of  actual  correlations  i.e., 
fully  disattenuated  true  score  correlations.  The  mean  of 
the  distribution  is  the  true  score  correlation.  The  main 
purpose  in  using  psychometric  meta-analyses  was  to  estimate 
the  true  score  correlations  between  the  various  dimensions 
of  job  performance.  Based  on  the  psychometric  meta¬ 
analyses,  an  intercorrelation  matrix  of  true  score 
correlations  between  the  different  dimensions  of  job 
performance  was  constructed. 

Confirmatory  Factor  Analysis 

The  series  of  psychometric  meta-analyses  conducted  on 
each  pair  of  job  performance  dimensions  establishes  the 
matrix  of  intercorrelations  between  the  dimensions  of  job 
performance.  The  intercorrelation  matrix  was  subjected  to 
a  confirmatory  factor  analysis  to  examine  whether  the 


60 


proposed  latent  structure  of  job  performance  explains  the 
true  score  correlations  among  the  dimensions  of  job 
performance. 

Confirmatory  factor  analysis  was  conducted  on  two 
matrices.  The  first  was  a  matrix  of  intercorrelations 
between  the  organizational  record  based  dimensions.  The 
second  matrix  was  formed  of  intercorrelations  between 
ratings  of  the  different  dimensions  of  job  performance, 
where  of  the  two  dimensions  being  correlated,  one  dimension 
was  rated  by  supervisors  and  the  other  was  rated  by  peers. 
The  reason  for  restricting  the  confirmatory  factor  analysis 
to  correlations  between  dimensions  (where  the  two 
dimensions  being  correlated  are  provided  by  two  different 
raters)  is  halo,  which  inflates  the  correlation  when  both 
the  ratings  being  correlated  are  provided  by  the  same 
rater.  The  CFA  program  (in  Basic) ,  developed  by  Hunter  as 
part  of  a  set  of  computer  routines  for  performing  a  variety 
of  analysis  on  correlational  data,  was  used  (Hunter,  1992; 
Hunter  &  Cohen,  1969)  . 

In  both  the  matrices  (one  based  on  organizational 
records,  the  other  based  on  ratings)  on  which  confirmatory 
factor  analysis  was  conducted,  I  tested  a  two  level 
hierarchical  model  involving  a  general  factor  and  specific 
factors  associated  with  each  of  the  dimensions.  I  also 
tested  a  three  level  hierarchical  model  involving  group 


factors  (where  some  dimensions  were  hypothesized  to  cluster 
together;  that  is,  correlate  more  with  dimensions  in  their 
clusters  than  with  dimensions  included  in  other  groups) .  I 
examined  the  residual  matrices  to  draw  inferences  about  the 
fit  of  the  hypothesized  models  (Hunter  &  Gerbing,  1982)  to 
the  empirical  data. 

I  formed  the  group  factors  based  on  my  review  of  the 
literature.  Care  was  taken  to  ensure  that  the  group 
factors  are  theoretically  meaningful.  The  important  point 
is  not  whether  the  group  factors  identified  are  absolute; 
alternate  hierarchical  models  can  (and  should)  be  tested 
(as  one's  theory  dictates).  The  purpose  of  testing  a 
hierarchical  model  is  only  to  test  whether  some  structure, 
could  be  realized  given  the  plethora  of  job  performance 
measures. 

The  parameters  estimated  in  the  confirmatory  factor 
analysis  of  a  two  level  hierarchical  model  are  the  factor 
loadings  of  the  various  dimensions  of  job  performance  on 
the  general  factor  as  well  as  estimates  of  the  unique 
variance  in  the  various  dimensions  of  the  job  performance. 
A  residual  matrix  indicating  the  extent  to  which  the  two 
level  model  explains  the  true  score  correlations  between 
the  different  dimensions  is  also  obtained. 


The  parameters  estimated  in  the  confirmatory  factor 
analysis  of  a  three  level  hierarchical  model  are  the  factor 
intercorrelations  among  the  group  factors  as  well  as  the 
factor  intercorrelations  between  the  group  factors  and  the 
overall  general  factor  of  job  performance.  Also  estimated 
are  the  factor  loadings  of  che  various  dimensions  of  30b 
performance  on  the  group  factors  as  well  as  estimates  of 
the  unique  variance  in  the  various  dimensions  of  the  job 
performance.  A  residual  matrix  indicating  the  extent  to 
which  the  three  level  model  reproduces  the  true  score 
correlations  between  the  different  dimensions  is  also 
obtained. 

The  extent  to  which  the  estimated  factor  loadings 
reproduce  che  intercorrelacion  matrix  between  the  various 
dimensions  of  job  performance  is  taken  as  a  test  of  che 
hypothesized  latent  structure  of  the  psychological 
construct  of  job  performance  (Hunter  &  Gerbing,  1982).  The 
results  of  the  analyses  are  presented  in  the  next  chapter. 


CHAPTER  IV 


RESULTS 

The  results  are  presented  in  two  sections.  In  the 
first  section,  I  present  the  true  score  correlations 
between  the  different  dimensions  of  job  performance.  I 
discuss  the  true  score  correlations  between  job  performance 
measures  based  on  organizational  records.  In  the  second 
section,  I  present  the  results  from  confirmatory  factor 
analyses  conducted  on  two  intercorrelation  matrices  (one 
composed  of  organizational  records,  the  other  of 
correlations  of  ratings  when  the  two  ratings  being 
correlated  are  provided  by  two  different  individuals)  to 
test  the  hypothesized  hierarchical  model  of  job 
performance. 

True  Score  Correlations 

Of  the  25  conceptually  distinct  dimensions  of  job 
performance  that  were  found  to  have  been  used  in  the 
literature,  5  were  based  on  organizational  records,  11  were 
based  on  supervisory  ratings,  and  the  remaining  nine 
employed  peer  ratings.  The  intercorrelations  between  the 
job  performance  dimensions  based  on  organizational  records 
are  presented  in  Table  1. 
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Insert  Table  1  about  here 


Of  importance  to  note  in  Table  1  is  the  relatively  high 
correlation  between  absenteeism  and  quality  of  job 
performance  (p  =  .48).  Also  of  interest  is  the  moderate 

correlation  between  accidents  and  quality  of  job 
performance  (p  =  .24).  These  findings  might  be  explainable 

in  terms  of  common  causal  antecedents.  For  example, 
perhaps  conscientious  individuals  produce  more  high  quality 
products,  are  less  likely  to  be  absent,  and  are  careful  to 
avoid  accidents.  Such  an  inference  also  meshes  with  the 
findings  of  Ones  et  al.  (in  press)  who  found  chat  integrity 
tests  predict  overall  job  performance  as  well  as 
absenteeism  and  accidents. 

As  could  be  seen  in  Table  1,  absenteeism  and  tenure  (p 

=  .33)  are  not  more  highly  correlated  than  absenteeism  and 
quality  (i.e.,  lack  of  errors;  p  =  .48)  or  absenteeism  and 
accidents  (p  =  1.0)  .  Though  a  true  score  correlation  of 
1.0  should  be  dismissed  as  a  chance  overestimate  (probably 
the  erroneous  estimate  resulted  from  a  small  sample  of 
unrepresentative,  biased  correlations),  the  results  are 
still  surprising  given  that  absenteeism  and  tenure  are 
hypothesized  to  fall  in  a  continuum  of  withdrawal 


behaviors.  A  related  point  to  note  in  Table  1  is  that 
tenure  and  accidents  have  a  very  low  correlation  (p  =  .07). 

The  true  score  correlation  between  tenure  and  quality 
of  performance  is  . 12  and  the  true  score  correlation 
between  tenure  and  productivity  is  .19.  Tenure  and 
accidents  correlate  only  .07.  However,  given  the  small 
number  of  correlations  in  each  meta-analysis,  the 
conclusions  have  to  be  very  tentative  only.  The 
correlation  between  productivity  (quantity)  and  quality 
indices  was  .37  and  this  estimate  is  based  on  13  samples 
involving  2,731  individuals.  Absenteeism  and  productivity 
correlated  .21  lending  support  to  Bycio's  (1992)  hypothesis 
that  absenteeism  disrupts  work  habits  and  lowers  individual 
productivity. 

In  general,  based  on  Table  1,  it  can  be  concluded  that 
in  forming  composites  of  job  performance  measures,  care 
should  be  taken  before  including  measures  of  tenure. 
Productivity,  absenteeism,  quality  of  performance,  and 
accidents  group  together,  encouraging  the  search  for  common 
causal  antecedents  such  as  conscientiousness  and  general 
mental  ability  to  explain  and  predict  the  various 
manifestations  of  job  performance. 

The  intercorrelations  between  supervisory  ratings  of  11 
job  performance  measures  are  presented  in  Table  2.  The 


true  score  correlations  between  9  job  performance  measures 
involving  ratings  by  peers  are  presented  in  Table  3.  Each 
correlation  in  Tables  2  and  3  represents  the  correlation 
between  ratings  of  two  different  dimensions  of  job 
performance  when  both  the  ratings  are  provided  by  the  same 
individual  (intra  individual  correlations) ,  These 
correlations  are  affected  by  a  general  impression  that  the 
rater  has  of  the  ratee  (i.e.,  halo)  that  affects  the 
ratings  in  the  two  dimensions  being  correlated,  thus 
inflating  the  correlation  between  the  two  performance 
dimensions. 


Insert  Tables  2  &  3  about  here 


The  correlations  in  Table  2  are  much  higher  than  in 
Table  1,  indicating  the  presence  of  halo  in  supervisory 
ratings.  These  values  give  a  picture  of  the 
intercorrelations  as  seen  by  individual  supervisors,  not  as 
agreed  on  by  different  supervisors,  and  not  as  they 
necessarily  are  in  the  real  world. 

Further,  when  the  same  rater  provides  rating  on  two  or 
more  dimensions,  the  intercorrelations  reflect  the  same 
perspective.  Arguments  have  been  advanced  that  raters  at 
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different  levels  (peers,  supervisors,  etc.)  of  the 
organization  have  different  perspectives  that  will  affect 
the  intercorrelations  (Sager,  1990;  Sager,  Nitti,  & 
Hazucha,  1993).  To  address  these  two  concerns.  Table  4 
provides  the  intercorrelations  between  measures  of  job 
performance  when  the  two  measures  being  correlated  are 
obtained  from  two  different  rating  sources:  supervisors 
and  peers . 


Insert  Table  4  about  here 


The  true  score  correlations  between  the  eleven  measures 
using  supervisory  ratings  and  the  five  job  performance 
measures  based  on  organizational  records  are  presented  in 
Table  5. 


Insert  Table  5  about  here 


The  correlation  between  the  nine  measures  of  job 
performance  using  peer  ratings  and  the  five  measures  based 
on  organizational  records  are  presented  in  Table  6.  The 
first  thing  to  note  is  the  large  number  of  empty  cells. 
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Further,  even  in  the  cells  where  the  true  score  correlation 
can  be  estimated,  the  sample  sizes  do  not  exceed  276,  and 
the  number  of  samples  involved  range  between  one  and  three 
(with  most  of  the  true  score  correlations  based  on  single 
samples) .  Because  of  the  small  sample  sizes  involved,  any 
inferences  based  on  Table  6  has  to  be  very  tentative. 


Insert  Table  6  about  here 


On  reflection,  it  is  not  surprising  that  most  of  the 
empty  cells  found  in  constructing  a  matrix  of 
intercorrelations  between  the  various  measures  of  job 
performance  were  between  peer  ratings  and  organizational 
records.  The  literature  in  I/O  psychology  has  examined: 

(a)  the  correlations  within  supervisory,  within  peer,  and 
within  organizational-record  based  measures  to  test  how 
many  factors  of  job  performance  emerge;  (b)  between  peer 
and  supervisory  ratings  to  test  whether  the  two  sources 
have  convergent  validity  and  to  assess  the  extent  to  which 
there  are  differences  in  the  perspectives  of  the  two 
sources;  and  (c)  between  supeivisory  ratings  and 
organizational  records  to  test  whether  supervisory  ratings 
sre  valid  (convergent  validity)  and  whether  supervisory 
ratings  are  more  highly  correlated  with  organizational 
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records  for  minorities  than  for  the  majority  group  (i.e., 
the  hypothesis  that  supervisors  when  rating  blacks  and 
women  pay  more  attention  to  organization  =.l  records)  . 

However,  there  seems  to  have  been  no  substantive  interest 
in  I/O  psychology  to  examine  the  intercorrelation  between 
peer  ratings  and  organizational  records.  Future  research 
should  address  this  limitation  and  examine  the 
intercorrelation  between  peer  ratings  and  organizational 
records  to  facilitate  the  development  of  comprehensive 
models  of  job  performance. 

Confirmatory  Factor  Analyses  Results 

Two  Level  Hierarchical  Models 

To  examine  whether  a  general  factor  emerges  from  the 
true  score  correlations  between  the  various  organization 
record  based  measures  of  job  performance,  the  5  by  5  matrix 
of  intercorrelations  (reported  in  Table  1)  was  subjected  to 
a  confirmatory  factor  analysis.  All  five  measures  were 
hypothesized  to  load  on  the  same  factor.  The  factor 
loadings  are  provided  in  Table  7  and  the  residual  matrix  is 
provided  in  Table  8.  The  residual  matrix  represents  the 
difference  (that  is,  the  discrepancy)  between  the  true 
score  correlations  reported  in  Table  1  and  the  correlations 
reproduced  by  the  hypothesized  latent  structure. 


Insert  Tables  7  &  8  about  here 


The  general  factor  explained  49.1%  of  the  variance  in 
the  five  job  performance  measures.  As  expected  (based  on 
an  inspection  of  the  matrix  of  true  score  correlations 
presented  in  Table  1) ,  measures  of  tenure  had  the  lowest 
loading  of  all  the  five  measures  on  the  general  factor. 
Surprisingly,  productivity  and  quality  had  lower  loadings 
on  the  general  factor  than  measures  of  accidents  and 
absenteeism.  This  is  probably  due  to  the  erroneously 
overestimated  true  score  correlation  of  1.00  between 
absenteeism  and  accidents.  An  inspection  of  the  residual 
matrix  indicates  that  the  true  score  correlations  of:  (a) 
accident  measures  with  measures  of  quality,  (b)  absenteeism 
measures  with  organizational  records  of  accidents,  (c) 
productivity  with  absenteeism,  and  (d)  accidents  and  tenure 
are  the  four  correlations  least  explained  by  the 
hypothesized  latent  structure  (that  is,  these  three 
correlations  have  the  largest  discrepancies  between  actual 
and  reproduced  correlations) .  These  results  indicate  that 
the  two  level  hierarchical  model  does  not  adequately 
explain  the  true  score  correlations  between  the  job 
performance  measures  based  on  organizational  records. 


To  examine  whether  a  general  factor  explains  the  true 
score  correlations  between  the  various  job  performance 
dimensions  that  involve  subjective  evaluations  (ratings  or 
rankings) ,  another  confirmatory  factor  analysis  was 
performed.  The  true  score  correlations  between  ratings  of 
various  dimensions  of  job  performance,  when  both  the 
ratings  being  correlated  are  provided  by  the  same 
individual,  are  affected  by  halo.  Halo,  which  reflects  the 
general  impression  the  rater  has  of  the  particular  ratee, 
is  confounded  with  any  potential  general  factor  across  the 
various  dimensions  being  rated  when  the  rating  source  is 
the  same  individual.  Halo  cannot  be  untangled  in  the 
analysis  of  the  true  score  correlations  reported  in  Tables 
2  and  3  (within  supervisors  and  within  peers) .  However, 
the  correlations  reported  in  Table  4  can  be  used  to 
investigate  the  existence  of  a  general  factor.  Here  of  the 
two  ratings  being  correlated,  one  is  provided  by  the 
supervisors  and  the  other  is  provided  by  peers.  There  are 
eight  dimensions  of  job  performance  that  were  rated  by  both 
supervisors  and  peers.  Based  on  these  ratings  an  8  by  8 
matrix  can  be  constructed  with  supervisors  rating  one  of 
the  dimensions  being  correlated  while  the  other  dimension 
is  rated  by  peers.  The  sample  size  weighted  mean  of  the 
corresponding  cells  above  and  below  the  diagonal  were 
computed.  The  values  are  provided  in  Table  9. 


Insert  Table  9  about  here 


Confirmatory  factor  analysis  of  the  correlations 
reported  in  Table  9  is  not  affected  by  halo.  The  results 
of  the  confirmatory  factor  analysis  are  reported  in  Tables 
10  and  11.  The  factor  loadings  of  the  eight  dimensions  of 
job  performance  on  the  general  factor  are  provided  in  Table 
10  and  the  residual  matrix  is  provided  in  Table  11. 


Insert  Tables  10  &  11  about  here 


The  general  factor  explained  54.9%  of  the  variance  in 
the  eight  job  performance  measures.  Ratings  of  overall  job 
performance  had  one  of  the  highest  loading  (.79)  on  the 
general  factor  as  could  be  expected.  Surprisingly,  ratings 
of  administrative  skills  also  had  an  equally  high  loading 
(.79).  Ratings  of  effort  and  job  knowledge  had  higher 
loadings  (.75  and  .76,  respectively)  than  ratings  of 
personality  (.69).  An  examination  of  the  residual  matrix 
shows  that  the  model  does  not  adequately  explain  the  true 
score  correlation  between  productivity  ratings  and  ratings 


of  effort  as  well  as  the  true  score  correlation  between 
ratings  of  productivity  and  ratings  of  problem  solving. 

Analysis  of  the  residual  matrix  as  well  as  the  percent 
variance  explained  by  the  general  factor  in  both  matrices 
(Table  1  and  Table  9)  indicate  that  a  two  level 
hierarchical  model  of  job  performance  involving  one  general 
factor  does  not  explain  the  true  score  correlations 
adequately.  Alternate  models  postulating  different 
groupings  of  the  job  performance  dimensions  based  on  the 
true  score  correlations  reported  here  (and  supported  by 
substantive  theories)  need  to  be  tested. 

Three  Level  Hierarchical  Models 

Based  on  a  review  of  the  job  performance  literature, 
five  group  factors  were  identified  and  all  job  performance 
measures  were  hypothesized  to  load  on  one  of  the  five  group 
factors.  The  five  group  factors  were:  Job  performance 
measures  focusing  on  (a)  productivity,  (b)  overall,  global 
evaluation,  (c)  withdrawal  behaviors,  (d)  interpersonal 
skills,  and  (e)  conscientious  behaviors. 

The  five  job  performance  measures  based  on 
organizational  records  can  be  grouped  into  three  of  the 
five  group  factors.  Absenteeism  and  tenure  will  load  on 
one  group  factor  (which  I  term  as  withdrawal) ,  whereas 
accidents  and  quality  of  production  are  hypothesized  to 
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load  on  another  factor  (which  i  name,  conscientiousness). 
Finally,  I  defined  productivity  as  a  separate  group  by 
itself  (Prod) .  Again,  to  reiterate,  the  definition  of  the 
three  group  factors  are  ad  hoc  and  interested  readers  can 
test  their  own  models  of  job  performance  based  on  the  true 
score  correlations  reported  in  this  dissertation. 

The  factor  loadings  of  the  five  job  performance 
measures  on  the  three  group  factors  are  presented  in  Table 
12.  The  factor  loadings  of  the  five  job  performance 
measures  were  as  hypothesized  with  the  exception  of 
absenteeism  and  accidents.  These  two  measures  did  not  have 
the  highest  loading  on  the  factor  they  were  hypothesized  to 
measure.  This  is  probably  due  to  the  estimated  true  score 
correlation  of  1.00  between  absenteeism  and  accidents  (an 
obviously  erroneous  overestimate) . 


Insert  Table  12  about  here 


The  intercorrelations  between  the  three  group  factors 
were  positive  and  substantial.  The  correlation  between 
prod  and  withdrawal,  between  withdrawal  and 
conscientiousness,  between  prod  and  conscientiousness  were 
.29,  .82,  .59,  respectively.  Subjecting  this  3  by  3  matrix 
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to  a  second  order  factor  analysis  (where  the  three  group 
factors  were  hypothesized  to  load  on  a  common  general 
factor)  resulted  in  factor  loadings  of  .75,  .84,  and  .96 
for  prod,  withdrawal,  and  conscientiousness,  respectively 
on  the  general  factor.  This  general  factor  explained  73% 
of  the  variance  among  the  three  group  factors.  This 
represents  a  49%  (23.9/49.1)  improvement  in  fit  over  the 
two  level,  hierarchical  model. 

A  three  level  hierarchical  model  can  also  be  tested 
using  the  8  oy  8  matrix  of  true  score  correlations  reported 
in  Table  9.  The  eight  dimensions  of  job  performance  are 
hypothesized  to  load  on  four  group  factors.  Ratings  of 
personality,  problem  solving,  and  administrative  skills  can 
be  grouped  together,  as  these  three  dimensions  of  job 
performance  reflect  interpersonal  relations  (Fl).  Ratings 
of  effort,  job  knowledge,  and  compliance  with  rules  and 
regulations  can  be  grouped  together  (F2)  as  reflecting  the 
conscientiousness  of  the  individual.  Ratings  of 
productivity  and  ratings  of  overall  job  performance  are 
treated  separately  as  the  third  (F3)  and  fourth  (F4) 
factors.  I  expect  the.  group  factor  involving  overall  job 
performance  (F4)  to  have  the  highest  loading  on  the  general 
factor. 

The  factor  loadings  of  the  eight  job  performance 
measures  on  the  four  group  factors  are  summarized  in  Table 


13 .  Even  though  most  of  the  eight  measures  had  higher 
factor  loadings  on  the  factor  they  were  hypothesized  to 
load  than  on  any  other  factor,  there  were  two  exceptions. 
First,  ratings  of  job  knowledge  hypothesized  to  load  on  a 
group  factor  {F2)  along  with  compliance  and  effort  (all 
three  presumably  reflecting  how  conscientious  the 
individual  is)  had  higher  loadings  on  group  factors 
reflecting  productivity  (F3)  and  overall  job  performance 
(F4) .  Second,  ratings  of  effort  clusters  more  with 
measures  of  overall  job  performance  than  with  ratings  of 
compliance  and  ratings  of  job  knowledge. 


Insert  Table  13  about  here 


The  intercorrelations  among  the  four  group  factors  were 
positive  and  substantial.  The  correlation  of  Fl  with  F2, 
F3,  F4  were  .70,  .44.  and  .90,  respectively;  the 
correlation  of  F2  with  F3 ,  F4  were  .83  and  .98, 
respectively;  and,  the  correlation  between  F3  and  F4  was 
.73.  When  this  4  by  4  matrix  of  intercorrelations  was 
subjected  to  a  second  order  factor  analysis  (to  test 
whether  the  four  factors  load  on  a  common  general  factor) , 
the  factor  loadings  of  Fl,  F2,  F3,  and  F4  on  the  general 
factor  were  .85,  .98,  .84,  and  1.01,  respectively.  The 


general  factor  explained  85.2%  of  the  variance  in  the  four 
group  factors.  This  represents  a  55%  improvement  in  fit 
over  the  tv;-'  level  hierarchical  model.  As  anticipated,  the 
group  factor  reflecting  measures  of  overall  job  performance 
(F4)  had  higher  loading  on  the  general  factor  than  any 
other  group  factors. 

Again,  to  reiterate,  the  three  level  hierarchical 
models  presented  and  analyzed  here  reflect  my 
interpretations  of  the  literature.  Researchers  could  (and 
should)  test  alternate  models  based  on  substantive  theories 
using  the  true  score  correlations  reported  here. 
Comprehensive  tests  of  three  level  hierarchical  models  have 
to  await  the  development  of  substantive  theories  in  the 
criterion  domain.  The  aim  in  this  dissertation  was  only  to 
estimate  the  true  score  correlations  between  the  various 
dimensions  of  job  performance,  test  two  level  hierarchical 
models  for  the  existence  of  a  general  factor  across  the 
different  dimensions  of  job  performance,  and  provide  an 
illustration  of  how  a  three  level  hierarchical  model  could 


be  tested  in  the  future. 
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CHAPTER  V 

IMPLICATIONS 

Convergence  Between  Sources  of  Evaluation 

Multiple  sources  (supervisory  ratings,  peer  ratings, 
organizational  records)  have  been  used  in  performance 
evaluation.  The  convergent  validity  of  the  sources  has 
been  the  focus  of  many  researchers  (e.g.,  Harris  & 
Schaubroeck,  1988;  Heneman,  1986;  Mount,  1984).  The 
correlations  presented  in  Tables  1  through  4  can  be  used  to 
test  the  convergence  between  different  sources  of 
measurement.  In  fact  a  more  reliable  and  better  estimate 
of  the  convergent  validity  between  two  sources  (e.g., 
supervisory  ratings  and  organizational  records)  for  overall 
job  performance  could  be  obtained  by  forming  the  composite 
correlation  between  the  measures  using  one  source  with  the 
measures  using  another  source.  In  forming  the  linear 
composite,  we  can  use  unit  weights  or  weight  the  measures 
by  their  loadings  on  the  general  factor. 

When  we  used  unit  weights  and  computed  (based  on  the 
intercorrelations  reported  in  Tables  1  through  4),  the 
composite  correlation  of  a  linear  combination  of 
organizational  records  of  absenteeism,  accidents,  quality 
and  quantity  of  production  (I  excluded  organizational 
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records  of  tenure  because  this  measure  correlates  low  with 
other  measures  involving  organizational  records.)  with  a 
linear  combination  of  supervisory  ratings  of  11  job 
performance  measures  was  .67.  Between  peer  and 
organizational  records,  the  correlation  was  .44  whereas  the 
correlation  was  .62  between  supervisory  ratings  and  peer 
ratings.  There  seems  to  be  more  agreement  between 
supervisors  and  organizational  records  than  between 
supervisors  and  peers.  Also  the  correlation  between  peer 
ratings  and  organizational  records  is  the  lowest  (of  all 
the  three  correlations  between  sources) .  As  such  we  can 
infer  that  organizational  records  reflect  the  supervisory 
view  more  effectively  than  does  the  perspective  of  peers. 

If  researchers  are  interested  in  capturing  different 
perspectives  in  performance  but  are  constrained  by  costs  to 
include  only  two  of  the  three  sources,  it  is  more  relevant 
to  collect  peer  ratings  and  organizational  records  chan  to 
collect  supervisory  ratings  and  organizational  records. 
Finally,  the  correlation  of  .67  between  supervisory  ratings 
and  organizational  records  inspires  confidence  in  the  use 
of  supervisory  ratings  in  organizational  research. 

Implications  for  Differential  Prediction 

The  findings  reported  in  this  dissertation  have 
implications  for  the  investigation  of  differential 
prediction.  The  positive  correlations  reported  in  Tables 
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1-4  encourage  the  search  for  common  predictors  or 
antecedents  of  performance.  That  is,  it  is  likely  that  the 
use  of  different  job  performance  measures  will  result  in 
the  selection  of  the  same  set  of  predictors.  Use  of  job 
performance  measures  that  are  highly  correlated  will 
probably  result  in  the  selection  of  the  same  test  battery 
(and  assignment  of  the  same  weights),  regardless  of  the 
criterion  used,  rendering  differential  prediction  unlikely. 

The  empirical  literature  investigating  differential 
prediction  using  multiple  regression  strategies  has 
provided  conflicting  results.  In  a  typical  study 
investigating  differential  prediction  using  the  multiple 
regression  approach,  different  regression  equations  are 
developed,  one  with  each  criterion  as  the  dependent 
variable,  with  a  set  of  tests  or  predictor  measures  as  the 
independent  variables.  The  purpose  is  to  examine  whether 
the  regression  weights  given  to  the  different  tests  are 
different  when  different  criteria  are  used. 

Schmidt  (1980)  reports  that  in  one  large  sample  study 
done  in  the  Army  it  was  found  that  a  job  sample  criterion, 
supervisory  ratings,  and  a  job  knowledge  measure  all 
resulted  in  the  adoption  of  the  same  selection  procedures 
and  assignment  of  essentially  identical  relative  weights. 
Oppler,  Sager,  McCloy,  and  Rosse  (1993)  found,  using  a 
sample  of  3,086  soldiers,  that  prediction  composites 
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developed  using  job  knowledge  tests  compared  favorably  in 
validity  to  those  developed  using  hands  on  tests. 

Despite  these  results,  researchers  have  continued  to 
focus  on  the  question  of  differential  prediction.  Some 
research  studies  claim  to  have  found  support  for 
differential  prediction.  The  contradictory  findings  across 
studies  examining  differential  prediction  may  be  the  result 
of  sampling  error  and  capitalization  on  chance,  which 
greatly  influences  the  results  when  multiple  regression 
strategies  are- used  to  develop  batteries  that  include 
correlated  predictors  (Hunter,  Crosson,  &  Friedman,  1985). 
The  estimated  regression  weights  have  large  standard  errors 
especially  when  correlated  predictors  are  used  unless 
sample  sizes  are  large  (Helme,  Gibson,  &  Brogden,  1957). 

An  alternate  strategy  to  examine  differential 
prediction  is  to  examine  the  intercorrelations  between  the 
job  performance  measures.  Differencial  prediction  of 
different  job  performance  measures  requires  that  the  rank 
ordering  of  job  applicants  differ  when  different  job 
performance  measures  are  used.  High  intercorrelations 
between  various  measures  lead  to  the  conclusion  chat 
differencial  prediction  is  unlikely. 

Recently,  a  hypothesis  has  been  advanced  (Campbell  et 
al.,  1992)  that  all  job  performance  measures  are  determined 


by  3  coinponents — declarativ©  knowledge,  procedural 
knowledge,  and  motivation.  This  hypothesis  holds  that 
dirferential  prediction  will  be  found  only  if  measures  of 
these  different  performance  components  are  included  in  the 
analysis  as  separate  measures.  The  low  true  score 
correlation  of  .20  (K  =  4,  N  =  386)  found  between 
supervisory  ratings  of  job  knowledge  and  peer  ratings  of 
effort  or  of  .09  (K  =  3,  N  =  325)  between  supervisory 
ratings  of  effort  and  peer  ratings  of  job  knowledge  is 
consistent  with  Campbell  et  al.  (1992)  hypothesis. 

However,  further  research  is  needed  to  actually  test  this 
hypothesis . 

Zarming  Composites  of  Different  Job  Performance 

Schmidt  (1980)  discusses  several  methods  that  could  be 
used  to  weight  the  job  performance  measures  in  forming  a 
composite.  The  Kellys  Bid  System,  the  Dollar  Criterion 
approach,  egual  weighting,  and  weighting  to  maximize  the 
general  factor  are  some  approaches.  Of  these,  forming 
composites  by  weighting  to  maximize  the  general  factor 
(Edgerton  &  Kolbe,  1936;  Nagle,  1953)  is  analogous  to 
weighting  test  items  by  their  correlations  with  total  test 
score.  It  is  appropriate  to  form  composites  by  weighting 
'-he  individual  measures  to  maximize  the  general  factor  only 
if  the  true  score  correlations  between  the  individual 
measures  indicate  support  for  a  large  general  factor. 


The  true  score  correlations  presented  here  can  guide 
researchers  in  forming  composites.  For  example,  in  forming 
a  composite  of  organizational  records,  care  should  be  taken 
before  including  measures  of  tenure.  Further,  Campbell 
(1990c)  says  that  core  job  performance,  demonstrating 
effort,  and  maintenance  of  personal  discipline  are 
important  job  performance  measures  in  all  jobs.  The  true 
score  correlation  (reported  in  Table  4)  between  supervisory 
ratings  of  productivity  (or  job-specific  task  performance 
to  use  Campbell's  terminology)  and  peer  ratings  of  effort 
is  .50  (K  =  2,  N  =  191);  peer  ratings  of  maintaining 
personal  discipline  and  supervisory  ratings  of  effort 
correlated  .56  (K  =  3,  N  =  378);  and,  peer  ratings  of 
maintaining  personal  discipline  and  supervisory  ratings  of 
job-specific  task  performance  correlate  .53  (K  =  1,  N  = 

164) .  These  high  correlations  suggest  that  forming 
composites  by  weighting  to  maximize  the  general  factor  is 
acceptable  in  all  jobs. 

Finally,  the  results  reported  in  this  dissertation  have 
implications  for  testing  competing  models  hypothesized  for 
the  latent  structures  of  job  performance  as  well  as  for 
theory  building.  The  true  score  correlations  reported  here 
form  the  building  blocks  that  could  be  used  to  test 
alternate  latent  structures  of  job  performance.  The 
positive  manifold  of  correlations  indicate  that 
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differential  prediction  is  unlikely,  encouraging  the  search 
for  common  predictors  and  antecedents  of  performance  and 
general  theories  of  work  behavior.  Future  research  should 
examine  the  correlations  of  the  general  factor  of  job 
performance  with  other  constructs  such  as  general  mental 
ability  and  conscientiousness.  More  specifically,  future 
research  should  examine  whether  group  factors  have  any 
relevance  (over  and  above  the  general  factor)  for 
evaluating  training  programs  and  other  human  resources 
interventions  in  an  organization. 
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Key  Words  Used  in  Searching  Electronic  Databases 

1. Job  performance/productivity 

2.  Theft 

3 . Absenteeism 

4 .  Turnover 

5.  Violence  on  the  job 

6.  Rule  breaking 

7.  Co-worker  relations 

8 .  Teamwork 

9 . Tenure 

10.  Alcohol  use/abuse 

11.  Drug  use/abuse 

12 . Tardiness 

13 . Lateness 


14. Quality  of  work 
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Journals  Searched 

1.  Journal  of  Applied  Psychology 

2 .  Personnel  Psychology 

3.  Academy  of  Management  Journal 

4.  Human  Relations 

5.  Journal  of  Business  and  Psychology 

6.  Journal  of  Management 

7.  Accident  Analysis  and  Prevention 

8 . International  Journal  of  Intercultural  Relations 
9 -Organizational  Behavior  and  Human  Decision  Processes 

10.  Journal  of  Vocational  behavior 

11.  Journal  of  Applied  Behavioral  Analysis 

12.  Human  Resources  Management  Research 

13.  Journal  of  Occupational  Psychology 

14 .  Psychological  Reports 


15. Journal  of  Organizational  Behavior 
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I . Organizational  records  of  quality  of  performance  (lack  of 
errors ) 

2 .Organizational  records  of  productivity 
3 . Supervisory  ratings  of  personality,  teamwork  etc. 

4 . Organizational  records  of  absenteeism  (lack  of) 

5 .Organizational  records  of  tenure 
6 . Supervisory  ratings  of  productivity 
7 . Supervisory  ratings  of  effort 

8 . Supervisory  ratings  of  overall  job  performance 

9. Peer  ratings  of  administrative  skills 

10 . Supervisory  ratings  of  administrative  skills 

II.  Peer  ratings  of  productivity 

12 . Supervisory  ratings  of  quality  of  job  performance 
13 . Supervisory  ratings  of  job  knowledge 
14 . Supervisory  ratings  of  absenteeism 

15 . Supervisory  ratings  of  problem  solving  and  leadership 
16 .Organizational  records  of  accidents  (lack  of) 

17. Peer  ratings  of  problem  solving  and  leadership 

18 . Supervisory  ratings  of  compliance  and  acceptance  of  autho 

19 . Supervisory  ratings  of  communication  skills 

20.  Peer  ratings  of  overall  30b  performance 

21.  Peer  ratings  of  personality,  teamwork 

22.  Peer  ratings  of  communication  skills 

23.  Peer  ratings  of  effort 

24.  Peer  ratings  of  job  knowledge 

25.  Peer  ratings  of  compliance  and  acceptance  of  authority 
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Table  1.  True  Score  Intercorrelations  Between  Job  Performance  Measures 
Based  on  Organizational  Records 


ERRORS 

PRODUCT 

ABSENT. 

TENURE  ACCIDENT 

Lack  of  errors 
(Quality) 

(ERRORS) 

.783 

Productivity 

(PRODUCT) 

p  =  .37 

K  =  13 

N  =  2,731 

.883 

Absenteeism  (Lack 
of ) 

(ABSENT. ) 

p  =  .48 

K  =  12 

N  =  1,290 

p  =  .21 

K  =  7 

N  =  1,825 

.763 

Tenure 

( TENURE ) 

p  =  .12 

K  =  7 

N  =  1,216 

p  =  .19 

K  =  7 

N  =  1,522 

p  =  .33 

K  =  26 

N  =  5,888 

.763 

Accidents  (Lack  of) 

(ACCIDENT) 

p  =  .24 

K  =  2 

N  =  1,061 

p  =  .39 

K  =  1 

N  =  975 

p  =  1.00 
K  =  3 

N  =  409 

p  =  .07 

K  =  4  -533 

N  =  806 

Note,  p  =  True  score  correlation;  K=  number  of  correlations  included  in 
the  psychometric  meta-analysis;  N=  Total  sample  size  involved  in  that 
psychometric  meta-analyses. 

^The  mean  of  the  internal  consistency  estimates  used  in  the  corrections. 
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Note-  p  =  True  score  correlation;  K=  number  of  correlations  included  in  that  psychometric  meta¬ 
analysis;  N  =  Total  sample  size  involved  in  that  psychometric  meta -analysis . 

^The  mean  of  the  intrarater  reliability  estimates. 


Table  3.  True  Score  Intercorrelations  Between  Job  Performance 
Measures : Peer  Ratings  Only 


ADM.  PROD.  PROB.  OVERALL  PERSON.  COMMUN  EFFORT  J.  K.  COMPLI. 
_ _ SKIL. _  SOLVE  _ ; _ _ _ 

Administ.  Skills 
(ADM.  SKIL.)  •7®" 


Productivity 

(PROD.) 

Problem 
Solving  & 
Leadership 

(PROB. 

SOLVE) 


p  =  .83 

K  =  4 

.55- 

N  =395 

p  =  .52 

p  =  .74 

K  =  7 

K  =  4 

N=1649 

N  =  202 

Overall  Job 

„  '  K=4  K  =  6  K  =  9 

Performance  n  =  491  n  =  554  N  = 

(OVERALL)  1345 


Personality 

(PERSON.) 

p  =  .60 

K  =  8 

N  = 

p  =  .76 

K  =  5 

N  =  321 

p  =  .62 

K  =  23 

N  = 

p  =  .81 

K  =  13 

N  =  2606 

.78^ 

1434 

4512 

Communication 

Skills 

(COMMUN.) 

p  =  .75 

K  =  2 

N  =  750 

p  =  .72 

K  =  1 

N  =  72 

p  =  i>3 

K  =  6 

N  = 

1644 

p  =  .09 

K  =  1 

N  =  72 

p  =  .62 

K  =  4 

N  =  1166 

.77^ 

Effort 

(EFFORT) 

p  =  .76 

p=1.00 

p  =  .9I 

p  =  .87 

p  =  .83 

p  =  .75 

K  =  5 

N  =  657 

K  =  2 

N  =  191 

K  =  6 

N  = 

K  =  5 

N  =  1341 

K  =  19 

N  =  2970 

K  =  1 

N  =  344 

.78^' 

Job  Knowledge 
(J.  K.  ) 

p  =  .74 

K  =  3 

N  =  494 

p=1.00 

K  =  2 

N  =48 

1029 
p  =  .89 

K  =  8 

N  = 

p=1.00 

K  =  5 

N  =  1374 

p  =  .83 

K  =  12 

N  =  2573 

p  =  .52 

K  =  1 

N  =  406 

p  =  .95 

K  =  6 
N=1338 

.78- 

Compliance  & 
Acceptance  of 

p  =  .90 

K  =  1 

N  =  164 

p  =  1.00 

K  =  1 

N  =  164 

1981 
p  =  .63 

K  =  1 

N  =  192 

p=1.00 

K  =  2 

N  =  328 

p  =  .99 

K  =  3 

N  =  378 

1 

p  =  .89 

K  =  5 

N  =711 

I 

authority 

(COMPLI.) _ _ _ _ _ 

Note .  p  =  True  score  correlation;  K=  number  of  correlations 
included  in  that  psychometric  meta-analysis;  N  =  Total  sample 
size  involved  in  that  psychometric  meta-analysis. 

^The  mean  of  the  intrarater  reliability  estimates. 


Table  4.  True  Score  Cortelations  Between  Berlorinance  Measures,  One  Rating  P'rom  Peers  the  other  Prorri 
Supervisors _ 


Table  5.  True  Score  Correlations  Between  Supervisory  Ratings  and 
_ Organizational  Records _  _ 


Organizational  Records 

Sup  ratings 

Lack  of  Errors 

Productivity 

Absenteeism 

Tenure 

Accidents  (Lack 

(Quality) 

(Lack  of) 

of) 

Personality 

p  =  ,34 

p  =  .26 

p  =  .33 

p  =  .11 

p  =  .44 

K=  12 

K=  16 

K=  15 

K=  13 

K=  13 

N  =  1290 

N  =  2201 

N  =  2753 

N=  1832 

N  =  2704 

Productivity 

p  =  .06 

p  =  .44 

p  =  .27 

p  =  .17 

p  =  .76 

K  =  7 

K=  10 

K=  12 

K  =  8 

K  =  4 

N  =  931 

N  =  1387 

N  =  1349 

N  =  975 

N  =  652 

Effort 

p  =  .39 

K  =  8 

p  =  .66 

K  =  6 

p  =  .54 

K=  10 

p  =  .09 

K  =  8 

p  =  1.00 

K  =  4 

N  =  1.366 

N  =  508 

N=  1615 

N  =  1383 

N  =  526 

p  =  .32 

p  =  .35 

p  =  .10 

p  =  .25 

p  =  .25 

Overall  Job 

K  =  20 

K  =  43 

K  =  37 

K  =  47 

K  =  9 

Performance 

N  =  4365 

N  =  8467 

N  =  13302 

N  =  23294 

N  =  2843 

Administrative 

Skills 

p  =  .34 

K=  1 

p  =  .39 

K  =  6 

p  =  .41 

K  =  2 

p  =  .23 

K  =  3 

1 

N=  100 

N  =  776 

N=  164 

N  =  204 

Quality 

p  =  .17 

p  =  .14 

p  =  .30 

p  =  .15 

p  =  .74 

K  =  6 

K  =  7 

K  =  8 

K  =  3 

K  =  3 

N  =  763 

N  =  674 

N=  1153 

N  =  681 

N  =  566 

p  =  .06 

p  =  .33 

p  =  .37 

p  =  .41 

p  =  .63 

Job  Knowledge 

K=  1 

K  =  5 

K  =  5 

K=4 

K  =  5 

N  =  100 

N  =  491 

N  =  567 

N  =  235 

N  =  923 

p  =  .36 

p  =  .21 

p  =  .09 

p  =  .11 

Absenteeism 

K  =  3 

K=  1 

I 

K  =  3 

K=  1 

N  =  428 

N=  133 

N  =  607 

N  =  86 

Problem 

p  =  .19 

p  =  .39 

p  =  .63 

p  =  .21 

p  =  .74 

Solving  & 

K  =  3 

K=  11 

K  =  2 

K  =  5 

K  =  5 

Leadership 

N  =  554 

N  =  1674 

N  =  291 

N  =  746 

N  =  860 

Compliance  & 

p  =  .18 

p  =  .33 

p  =  .12 

p  =  .41 

p  =  .38 

Acceptance  of 

K  =  3 

K  =  5 

K  =  3 

K  =  6 

K  =  6 

Authority 

N  =  381 

N  =  711 

N=  1091 

N  =  1059 

N  =  1471 

Communication 

p  =  .25 

p  =  1.00 

p  =  -0.02 

p  =  .57 

Skills 

I 

K=  1 

K=1 

K=  1 

K  =  3 

N  =  40 

N  =  142 

N  =  100 

N  =  566 

Note,  p  =  True  score  correlation;  K=  number  of  correlations  included 
in  that  psychometric  meta-analysis;  N  =  Total  sample  size  involved  in 
that  psychometric  meta-analysis.  Interrater  reliabilities  were  used 
to  correct  supervisory  ratings;  internal  consistency  measures  to 
correct  organizational  records. 
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Table  6.  True  Score  Correlations  Between  Peer  Ratings  and  Organizational 
_ Records _ 

_ _ —  Organizational  Records  _ 

Peer  ratings  Lack  of  Errors  Productivity  Absenteeism  Tenure  Accidents  (Lack 
— - _ (Quality)  _  (Lack  of) _  of) _ 

p  =  .09  p  =  .14 

K=l  K=1  I 

N  =  149  N  =  149 


Personality 


p  =  .19 
K  =  2 
N=  172 


p  =  .18 
K  =  2 
N  =  172 


Productivity 


p  =  -.13 
K=  1 
N  =  72 


p  =  .47 
K  =  2 
N=  174 


I 


Effort 


I 


Overall  Job 
Performance 

Administrat 

Skills 


p  =  .23 
K=  1 
N  =  72 


I 


Job 

Knowledge 

Problem 
Solving  Sc 
Leadership 

Compliance  & 
Acceptance 
of 

Authoriity 
Commun . 
Skills 


I 

p  =  .17 
K  =  2 
N=  144 


I 


p  =  .14 
K=  1 
N  =  72 


p  =  .66 
K  =  3 
N  =  276 

p  =  .07 
K=  1 
N=  102 


I 

p  =  .37 
K  =  3 
N  =  243 


I 


p  =  .15 
K=  1 
N  =  72 


p  =  .39 
K=  1 
N=  149 


I 


I 


I 

p  =  .67 
K=  1 
N  =  149 


I 


I 


p  =  .34 
K=  1 
N=  149 

p  =  .84 
K=  1 
N=  149 


I 


I 

p  =  .44 
K=  1 
N=  149 

I 


I 


I 


I 


I 

I 

1 

I 


I 


p  =  True  score  correlation;  K=  number  of  correlations  included 
in  that  psychometric  meta-analysis;  N  =  Total  sample  size  involved  in 
that  psychometric  meta-analysis.  Interrater  reliabilities  were  used 
to  correct  Peer  ratings;  internal  consistency  measures  to  correct 
organizational  records. 
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Table  7,  Factor  Loadings:  Cofirmatory  Factor  Analysis  of 
Organizational  Record  Based  Measures 


Performance  Dimension 

Loading  on  General  Factor 

Lack  of  Errors 

.64 

Productivity 

.63 

Absenteeism  (Lack  of) 

00 

00 

Tenure 

.50 

Accidents  (Lack  of) 

.79 

Table  8.  Residual  Matrix:  Confirmatory  Factor  Analysis  on 
Organizational  Record  Based  Measures 

ERRORS  PRODUCT  ABSENT.  TENURE 


Lack  of  errors 
(Quality) (ERRORS) 

Productivity 


(PRODUCT) 

.03 

Absenteeism (Lack 

of ) (ABSENT. ) 

.08 

.34 

Tenure  (TENURE) 

.20 

.13 

.11 

Accidents  (Lack  of) 

.26 

.11 

-.30 

(ACCIDENT) 


Table  9.  Coi relations  of  Ratings  subjected  to  Confirmatory  Factor  Analysis 
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Table  10.  Factor  Loadings:  Confirmatory  Factor  Analysis  of 
Ratings  Based  Measures 


Performance  Dimension 

Loading  on  General  Factor 

Personality 

.69 

Pi  oductivity 

.71 

Effort 

.75 

Compliance 

.71 

Problem  Solving  Skills 

.72 

Job  Knowledge 

.76 

overall  Job  Performance 

.79 

Administrative  Skills 

.79 

Table  11.  Residual  Matrix  From  Confirmatory  Factor  Analysis  o£  Ratings 
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Table  12.  Factor  Loadings  of  the  5  Organizational  Record  Based  Job 

_ _ ^ _ Performance  Measures  on  3  Group  Factors _ 

Variables  G1 -  prod  G2 -withdrawal  G3 -CONSCIENTIOUSNESS 


Productivity 

(PRODUCT) 

.94 

.27 

in 

in 

Absenteeism  (Lac>c  of) 

(ABSENT. ) 

.22 

.74 

1.07 

Tenure 

( TENURE ) 

.20 

.74 

.14 

Lack  of  errors 
(Quality) 

( ERRORS ) 

.40 

.41 

.73 

Accidents  (Lack  of) 

(ACCIDENT) 

.42 

.73 

.66 

Table  13.  Factor  Loadings:  Confirmatory  Factor  Analysis  of  Eight 


Job  Performance  Ratings  Loading  on  Four  Group  Factors. 


SUPERVISOR 

RATINGS 

G1-INTERPERSONAL 

G2-CONSCIENTIOUSNESS 

G3-PROD 

G4-OVERALL 

Personality 

.77 

.54 

.38 

.69 

Leadership 

.82 

.66 

.15 

.70 

Administrative  Skills 

.89 

.55 

.57 

.86 

Compliance  & 
Acceptance  of 
Authority 

.53 

.83 

.56 

.60 

Job  Knowledge 

.46 

.78 

.91 

.88 

Effort 

.  64 

.77 

.46 

.79 

Productivity 

.41 

.77 

.93 

.68 

Overall  Job 
Performance 

.70 

.77 

.57 

.78 

